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ABSTRACT 


Micemeioemontatlicn Of OMB Circular A-76 to secure ccst 
Savings ty federal agencies has resulted in increa 
contracting out of Commercial Activities (CA) formerly 
BesrOnrmeca by ene gcvernment workficcce. This the 
paomecenercund Of the A-/6 rolicy and describes its require- 
Setese LD lementaticn of the CA orogran, pa 
NAVFAC, le investigated in detail. An A-76 emphasis 
exfective gcvernment cperations has fostered the aprlication 
cf statistical quality assurance techniques for CA service 


contract adainistraticn. These are presented, as well as an 


overview cf extrepolated deductions based on sampling +ech- 
niques. Finally, improvements in organization staffing and 
Structure ars examined. The study zctecommends increased 


usage cr statistical quality assurance, more comprenensive 
piarning and budgeting of inspection rescurces, and ths 
feumecticn cr coneezaliged CA contract adminstration organi- 


Zacenens at ¢he Lisld activity level. 
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A. GENEKAIL COMMENTS 


The Cffice of Management and Budget (OMB) Circular Wo. 
A-76 requires federal agencies to inventory and review all 
Ge theiz Ccmmercial Activities (CA). A Commercial Activity 
is defined as cnée which is managed and operated c¢ither by 
the gevernment cr by a private firm and which prevides 2 
BPeoate. Gr service tc a federal agency. Unless the activity 


is an inherently governmertal function which is required to 


(i) 


retained in-house, it must be subjected to a rigercus 
cost ccmparison to determine the most economical way to 


perform the work. Ir the cost study shows that it is cheacveéer 


f 


Dene kxne GOvVGainert tc perform the work, in-house rescurc 
maieere §6used, HOWweven, Lif it %issmore sconemicel for “he 
beumat> Sec:cr t9 p=evids the product or services, the <sunc- 
Pron 2s Ccntracted out. 
Mires l, the f£eascr 


a 
Seevices c£ a ccmmercial nature, including maintenance ct 


-- 


GOVeniANeteens0eN= p52. 5° bi 1 Pie, Jor 


equipment, military base operations, and Eacility suepe=s 
such as housekeeping, SeGlaescy , and food sérvices. 


(Ret. 1]. Currentiy, an estimated 409,000 federal enpicysces 
Pe=fC=m an-heuse CA functions valued at $20 biilion. As 
federal agencies move to comply with A-76 policy, he level 


Semeccrecaculng CUx Of CA Etnctions is expeczed <=c grow 
Seemenaewe yy. £2 LS imperative that this implementaticn de 
Svoe-Seaciec and well thought out in order to chisve beth 
Baximum ¢ccenoemy and efficiency of operaticns. [{Ref. 2] 
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Be SITULY OBJECTIVES 


The principal objectives of this research 
=<omedewens eDpplicaticn of statistical quality co 
Service ccntracts and evaluate improved 4a Dees 


administzraticn organizations. 


Cw. SCOFE AND ASSUMPTIONS 


Boswdeugh the CCMercial Activizkiscs policy is applicabls 
<>. 4«6all federal agencies, this Seitiy @@4ili focus o 

Lepartment ct Defense (DOD), wath oarticular emphasis on zhe 
Névy in gen¢erai and the Naval Facilities Engineering Command 
fawn AG) “2h particular. It is assumed that the reader is 


Bttelage Wich the acquisiticn process within the Department 


De. RESEARCH METHODOLOGY 


Me ieimaal Jiteratwre search was conduczed to review 


PSone het luctlOns, Teaqulations, policy guidances, indus- 
Sea! tet=rature, and G@eperts applicanle to A-76 Laflenenta- 
ert). Interviews and discussions were conducted during 
meeeem yo a2ss Wlth personnel at the Eollowtag organizattons: 


Vo Hevel Supoly c 
eee vel Ceworks Cent: 


3. Western Division, Naval Facilities Engineerin 


LQ 


GCeptwerd, San Exuno, CA; 


RH. Naval Weapons Station, China Lake, CA; 
ee WEteewOGKkS Canter, San Diego, CA; 
Seeeemmeeongineer Corps Officer School, Port Huenene, 


Cx: 
Poperceellam Ait force Base, Sacramento, Ca; 


Ses eVeseAa:t ©eCrce sase, Failrtield, CA; 








Souerpurcele Works Lepartment, Naval Shipyard Long Beach, 
CA; and 
TO.  2URLcC Werks Leparemen®, Nawal Pestg@aduazte Scnoci, 


Monterey, CA. 


moemeectonral telerhcniec conversations were held @eyach 
Dio tweens ©OlleCwing headquarters orsganizaticne: 
foe sovaratiatscrial Command, Washington, D.C.; 
2. Naval Facilities Engineering Coamand, Alexandria, VA; 
3. Naval Supcly Systems Command, Washington, D.C.; 
peeeemee Ge Of Federal Frocurement Policy, W#o2SH2nG=Ca, 
Bee. ; a4hb 
5. tees cna lL AeIcnautics and Space Manin s es aroen, 


Washington, D. C. 


TRESIS CRGANIZATICN 


Esecen.cS Khe auenors'! objec ti 
9 u 


iscuss2e the scope, requirements, and 2 em 
Bavoume craters: Lif identifies key issues in CA Conversions, 
=  Chepete IV 2xemines the applecation of statistical 
mem ciemer ole. > CA CON TEacts. DME «eke 2 AO . Cla tuers 
ress the planning, DUdGee3og era vdssi gr cs che ssavice 


Meccreecdten i Sika =2cn Organization. Finelly, oe vay 
Senta che Scucy's tecommendazttens icr improvements ia CA 
mecca Geyer Sions. 


i 








Bie eGoveanmene'’s policy for not competing with the 
P@awere, S=Ccor was tirst established durin President 
Eisenhower's administration. Lats, sthespultean of =he 
BFudgez (ECB), the predecessor of the Office of Management 
ard udg¢= (OHB), rublished Bulletin Number 55-4 wnicn 


foes she genera l peae ly Of tne administration «haz the 
Federal Gcvernnent i NOT start OF carry on any 
commercial activity to See a scate Co Or Decule for 
Bec wOWN US2 i such a product or S¢ivice can 52 pr ee 
Ee M,C ee vars eos eos Piecudl mead inet we otel ness enan = 
nels HMMGeOG=ORS =O <chis pclicy snail” be made py the 
head of agency only wher2 it is clearly demonstrated ir 
SPermecas-amadt Lt iS not in the puolic interest tic 
Deoeurc Suca pacotcrs Or services from private enter- 


prise. {[Ref. 


Stet See gd “CO med "the 

Govertnmen='s general policy of relying on the orivate enter- 

Pec -@movyewewe ce SUPEly its neecs" [ Ref. 4&4] but it aiso 

PeeeceonsZcau sONeG instarces Where "At ls ia the fational 

Bee bese sG= <26 GOVEINnMert <O prcvids directly th2 preaucts 
Gc 


zt uses." [Ref. 4] 


Tne basic policy underwent another major chang? with 
oem eee temece Of GMS Ciscula= A-~-76 (revised) aaced 29 Mazch 
179 . Unlike che previous statements which cnily stressed 
GOvVern@er< reliance cn privat> enterprises, the new Eclicy 


Heer eemaguea' na prEanciples. ([Ref. 5] 
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my 


a. piey OR nS PTs fees Saosege ne Government's busi - 
Meese = bee «CeeDeS =n business. Where private sources 
a-° avablable, <ney. Shoul 4 De Vicor oWecOoLirs: to prcvide 
the ccrmer rola G= in@ustesel Gqceds ana Services nee ded 
by the Government to act cn thé pu bixze’s behalf. 

Db. Rétain Certain Governmertal Functions In-House. 
Cope sUGe-20RS are inherently governgenzal 2h nature, 
being sc inti@ately related to the public interest as te 
Mandate performance by Federal employees. 

Ge Aim for Econory; Cost Comparisons. . When private 
perfcimance is feasible and EO ovetridwng factors 
fegqeueese “n—-house periormance, =ne American fsopiz 
deserve and Se the most cconomical performance and, 
theregrcre, 2GO0LOUS COmmar=SoOr Of CORtraer Costs versus 
an ECuse ccs Shculd 53 used, when appropriate, <c 
decide how the work wili be doné. 

HE chPpcre <h= ney emendasis on “che eccencay of 
qocverngent Sn AouSse performancs compared TS private 
Syeerececcon “perftco mance, a Cose” Comparison Hermadbpecok was 
issued 4S a supplemert to A-76. Be year later, in April 
T9380, <h2 LTepasxment of Detense (DCD) 1Ssu¢ec 2tS Ccwn Cost 
Comspartzecn Rendpcok (DOD 4100.33-H) tO provide even Ror? 


a 

e d 

@ the Ccst CeEMpearisoas ana to improve their 
See weecyrena validity. SC@S gezetai g u 
Peer. 5} 


Seek oe me «GCVeErrsert and clogmarctal coSts must cs Easea 
6n the same scope cf work and the same ievei of 
cesricimace. 

Dimes cemacss cost factors wail b= based om <“«n€ Ccst 
Cemeat-son danchbeok. 

eyercum cC@cxsS are =O de used =S tht waxiw@idme extent erec- 
wucal: 

aq) For werkioceéds cf a ccntinuing na ure, Prose ced) o> 
remeweable cpticns should b2 requested from the ccna- 


ssactcr CO mindaiz> Duy=ins. 


W 
ea 
~ 
Mm 
4 
<j 
{+- 
O 
fh 


Gist ame cinder $100,000 annually shewlc «be 
e 





me —8ecewcee. Cemparzson will use a” rate cf 10 percent ter 


annum’as the cpportunity cost of investmernzs and of 


peewee. Procc<eds frogttbe pocsnggal sale cf carital 
assets. 


As a nesult cf the increased emphasis cn contracting 
OU=, COngtess haS repeetediy |expressed its concarn accut the 
S@eple@enceticon of the A-76 policy. Da eeliscCavey sck 9197 2 (Py 
78), Congress placed 4 one year moratorium on vircuall, 
pemmecGuemecreimrgG Cut CE A~76 Lunctions if such a cecnvarsicn 
would displace a Government employes. [ Ref. 6] At the start 
erent Y¥ a; Congrese institutes 2 Six MORtCT §OFazC= 2m On 
G=~-"ain) A-76 functicrs. BUehouga =he  MO0S¢" =Secen=s Laan 


Miece are alse two Public Laws which gauaide COD's 
a=Caisitca =<C convert. Ehe Overals management of DOD 
perscnm]=l resources is contained in DOD*s Avdproprsétion 
mieaCriza-icn ACE, eee CIs Os NNVCh S55 -pooveded it 
eppencrx A. hes ACt Sequices chat DOD “Use the Il2east 
CGoqesemmerOose SanNDCWer thas 2S Consistent with sil27421 
EeQuUiremeres and other necds cf the Dapartment cf Sefonse. 


a O 
tjzan personnel ceilin 
BEOVides Congress sop 


Ss 
SOR ¢ Gee cee tens, and reports in a timely Nanzec. 
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{Ref. 9] Thes® two prcvisions are considered to be perm 
laws whick will remain in effect until changed by 


legislaticn. 


ie SCCPE OF A-76 
1. Exclusions 


Mucmpuewistens Of OMB Cizrcilar A-76 do nct asply to 


nie wtclmemeng Gaeegqories and situations: f{Ref. 5] 


WCE EGeEN race: ng Out Of personal services thax would 
result in an employer-employse relationship. 

Ppeericum@agoa system acguisitions governed by OMB Cizcular 
A~1C9. 

en ER areyra Gemiscact cr SUPDCES ~S2rvaCces ahnich iene s 
censulting sexzvices, studies and analysis, and prefes- 
Sicnal and management support services. 


G) Whenever implementation would 


(D 
) 
O 
ry 
ct 
rf 
pv 
4 
bet 
ct 
Q 
j— 
1v 
x 
O 
is 


Pmectcratcieuwich “he terms of any treaty cr interna- 
ticnal agreanent. 

Slee em cas aeetivicy ils perform 
meres es LOT EaTOLleS, C= posssssicas. 

BEC T ees Catets and Services are cocbtained frem ccher 
mederal agencies which @r¢ authorized cr required by 


Laweeee bub sash the a. 


Wat °Or 0l 22 NOD leeaZa Sion. 


(4) 
fa 


gj) In times of decia: 


ct 
m 
b4 
hes 


In addition, A-76 recognizes tha 
mental functions must be perfermed in-house These gov2in- 
Mentai functions fall into three main categories. The first 
categcry is the discretionary exere f Government 


@autnerity. This includes: [Ref. 5] 
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iv etgaseons, PreSecutions and Other judicial furc- 
tiIche; 


b) management of government programs requirin 


73 


g 
judgements such as directing the rational defenses; 


C) management and direction of the armed serv 


t? 
Q 
(D 
"A 


ay) Gencuecing of f£créign relations; 

2) selection of thee Pel @aictlcs 4 

ft) direction cf federal employess 

g) regulation of natural resources such 4s the use of 
é 


3 
ceans, and inland watsiways; 


Pp 5 
h) ditecticn of intelligence ard ccunter-inte=llicence 
operatio 
eguiaticon of industry and commerce. 


The second categery of an inherently qovernmnental 


BPeCt LOX “2nvolVeS Menetary =ransactions and entitlements 
such as benefit prcgrams; Geax Secoltece=en. and =sverue 
ims DuESeMebaeSs control of financial accotnts and the admta- 
moeesat.cn cil ‘puleiec trusts. The last type of government 


PIMGZIOn a5 tae COMduct of research and development at test 
BMeGewescice and the cpertationr 

laporatorics, Bese WesanGes, [est gue oraw.  ~andey- Siu ts.< 

{Ref. 5] 


PReeeeantemolel Netw 1 eS Subjec= 0 A- 76 


Mamcms@nvicemee@e-Vvity 2S gao9c specifically ¢xclu 


qd 
mmOm PONE A-~7o 2S prevactsly discussed in Section 31 cf this 


Siape co, seera 25 2Ot am anherentiy governmental functica as 
defined in Section B2, then it is classified as a Commercial 
AGtA VEE Y. ho wecwm@e nc A =o OMB A-76 DrOVides approximately 
Cn hundred exanples of Ccmmercial Activities for fiftcen 
Git ecoeent Service caveqories. Sueneecommercial Activities 
Bay be operated and managed either by =he governmenux agency 
Or 32 private commercial business. 
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In 
O 
: 
8 


ent Oreration of 


I 


Commereras ACTiVvis 


In-heuse perfcrmance cf Commercial Activitiss cannot 


Bomgicmernca Solely cn the basis that an activity sufrorts 


Cee NvVemvesva Classified progran, or is part of én agency's 


bkasic 


Missicn, or that there is a possibility of &@ strike by 


contract employess. Government operation of a Cemmertcial 


4 


ReacivVanyecaneonly be auzthorized under one of tha Sollcwing 
Ccomait2ons: fRef. 5] 


a) 


D) 


Now SeetSilacctory Commercial Source Is Available. 
Government operation is permitted whenever it can be 


@ocunenzted that ez«her: 


ae) There 1S no private commercial source capable of 
providing the needed service, or 

cae) That the us2 of a privats source would cause 
unacceptatle delay or disruption of an essential 
frogram. The required documentation must be 
defacto Gmwin terms Of Cost, time, end pericrmancs 
measures. ihewds SLiperonmmuce, He Ot. a Jas cing 
rature and not just temporary. 


Naticnal Defense. 
) 


j-- 


Government operation by military personnel is 

permitted whenever: 

imem—ec=conne! ate Ueilized in or subject £3 

depiieytens in a2 dtzrsect combat or ccmbat 
service suprort rol 

Pelee aGtavacy 1S essential for exclusively 


ieee Gary etraining. 


DV ccweaweery 25 LPequized => provide afporo- 
Pome eenwec-k 2SSi102NE1CS fee carese> proegres- 
S1On cGr a rotation base fcr overseas Of sea 
*o shere assignments. 
zc) Government oneraticn of a depot or intermediats 
jJevel maintenance facilicy may be Jjusticied to 
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PUowecedqmeready and CoOntsolled source of <=Sch- 
nical competence and resources necessary ts zncex 
military contingencies. 

GuueLower COST. Liewowe Of =neeDecececaing conditions can 


be met, gcverrment operation cf a Commercial Activiry 
can only be authorized when a comparative ccst 
analysis, perfermed in accordance with A-76 and the 


Gece GCompatisor Handbook, shows that in-house crera- 


is 
f. 


ieos- than 25 mete. wets cbhtat ned 


ct 
O 
ct 
fv 


1cm has a lewer 


=x 


2 


een 


4%) 
£2 
= 
m@ 
}~ 
{-'- 
rt, 
+? 
7h 
Qu 
TO 
4 
< 
) 


ao SOULC =. 


[eemavemeory Cl Gemmercial Activity Funetions 


SceeeCteecmias A-J6 Tequared sacn agency =c compile 
- 


and update annualiy a complete inventory of ail Ccmmercial 


ee 
| N OMB ER OF ANNUAL OPERATING MAN-YEARS 
| Server er CGAgmUMETZONS COSTS {BILLION $} (THUCUSANLTS) { 
| 
| ARMY 2,041 Pal 133 | 
| NAVY/XARINE Op aoc S74 278 | 
| Ape ene © 5,0 24 oneo 260 | 
fr) OCLESE CCD 4E4 mae 18 | 
ee. fa 
! Cea Wee eo 1720 6 89 | 
a 
i a is a ies = ee ee a 
Figure 2.1 DOD Inventory of CA Functions. 
Meee rtaeae subjscce te 2tS provisions. The 32 ia eee cop 
HevencOry was completed in August 1980 and is summarized in 
Hegucesz.1. { Ref. 10] 
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evens ceuet2on 4100533 cegquires that a ccmclets 


Bevzaew Ci all CA functicns Ee completed during FY 89 through 
rc 


BY 84, Subsequent reviews art required at isast once every 
Five years. In the event that the circumstances supporting 
the initial approval are not subject to change, then subse- 
guent.reviewS may be waived by the Assistant Secretary cf 
Defense (Manpower, Reserve Aifairs, and Logistics). 

The purpese cf the réview is to determine whether 
Mien pm&scaocnht Methcd of performance should be continued cr if 


Mace raunce2On Should be designated for accst comparison 
analysis for possible change in method of performance. ey 
Making this determination, the cri specified in A-76 
and explained previously in «his chk are used. Pe cne 
mimeracn iS ~resentl Cent raceed, SOUGHn =sStintare Cis-bs 
Seok cc  perionm the work in-house is prepared. ite. S 
ceee@eaesd in-heuse cost _s not less than contract perfor- 
Mamece OY) 10 percent Cr personnel cost ana 25 percent cf ccst 
GEmOwNelSntE SF equactment and facilities, then the functicn 
Memaius ccn*tracted out. AOWGVeEL, “be ne; Te xels nocd 2x2scs 
mise n-acuse performance would meet the cost dirtferential 
Me eerea,  2ermal COst Comparison analysis must be perftcrmed 


tc determine tae chearer method. 


2. Levsiopment cz Stazement of WoO=k 
iiem—vEecmoat len, OL the Statement cf werk . (SOW) ‘as 
Shen Om che Mest critical Peps 2a) cs efleceive trplementa- 
pee CE =he A-76 policy. ie ose gn 42 eds seccly lLmpect 


Docmemae eo MOL c De SOliGicah ions, the cost comparison precceéss 


ct 


and subsequent rerfcrmance either by in-nouse péerscnunel or 
Dy ccntractcr employees. The SOW nust establish the govern- 
BMent's actual tininum requirements for performing the 


service. These stancards are the same regardless of wnether 











eae “Work is perfcrmed by the governmen® Or . DY Saeene 
Gem <~ractcr. 

Pics sOtTGONSTI’cuteS the Sspecit@eations fcr ke 
Gentract. iPims Cul be Stee ectentiy comprehensive, 
expressing all requiremerts in a clear, unambigucus and 
concise manner. Zt should describe all dutéés, tasks, 
responsibilities, and frequencies of performance. The SOW 
sShouid be performance oriented and specify what is ‘to be 
done withcut prescribing how it must be dons. Megeoap se. 22 C 


procedures ars required, the government bears the risk «hat 


compliance ae 11 result Sta UNacecepta bis pertcepence. 
However, 2£ the SOW establishes the minzmum acceptable 
guality level (AQL), then the contractcr assumes full legal 
liability fer meeting this standard. 

Along wit <he SOW, aquality assurancs plan is 
required [{kef. 11}. This plan sets the surveillance 


BeaUcae@endtS ard precedures fcr the government's quality 
Eo u 


assurance evaluators. The quality ass 


Q P 

ensure that edequate performance is achieved and establishes 
te o ie 
S 


the mechanisms tor the administration e né service. 
Seeetees L¥ end V will examine these issues in greater 
4. Erzpace and Audit In-House Estimazs 
mee POp- AUcthCCiza=i1oOn Act, 1981 requires that 
Govecl omen. in-house estimates be ovased on the “most ef fi- 
Siemon ems . SLL eCia Ve Organization fcs perficrmaace"™. 
fee c 319 |. DOD Peeeaucticn 4100.3348 Gasecks chat : 
{Ref. 12] 
Peeueag-ncy ©heuid cheure thas Government Operations are 
Organized and staffed Cer, =he (Wiese TCIeh: pel ror= 
mance. Tc «he SxXtéent pra ~cable and in accordance with 
agency manpows and personnel sgula~icns, agenci¢es 
sncula pre Bde reviews’ Gig ered | cz reular With acces 
nanagemen= Cevigws SaameGoOrGantzazlLOns fOr accomplishing 
Piemween @622 =rticiencly,s«e#hen teasibie. 


ZZ 








MiewaCz-Vary is not required to achieve this most ef 
a. (TO EatormetO a Ceostmicompazison study bux iz 
BHS= USS the MEO as the basis for the gove 


estimate. 

fter the in-house ¢stimate is prepared, based on 
the estarblisned Statement cf Work and uSing the most eifi- 
CLeEnt 


A 
Ec - hh °C 
ECan Z7eaeren, 2t°5S regqud@ed to be eudited. The audit 
r D a m 
€ 5 a 


° 
is performed by n indevendent audit agency, ncrmailly the 
Naval Area Audit Service, and must be szarzed 120 days prior 
Bom ideOCSr ing. Retell J obycaddawon ~C¢ “ensuring cso per 
Gemoliance) wee=h A-7€ and the Cost Comparison Handbcock, ths 
Audit Service reviews and approves the proposed SOW and the 
BLO. 
9. sciscitation and Evaluation of Contraccor Proposals 
Once the in-neuse estimate aS Deen apprcved Seal 
Pete Gr elLOcosals will be solicited. Alehough competitive 
Pefeeceac-cnSs On a firm’ fixed price basis are ‘the preterred 
@echie@ OL Cemtracting, fetmal adverteésing and other pri g 
arrancements may be approved in raré instances. {Ref. 14] 
Meme Geeie COMRETECting Cr=aicer opens the bids or ccnpletes 


Ss a CCe bed DL emcen tric: 


Cr xX 
Pimice@geceWs> ©CCSt,  <ehen tne fenmormance py the g 
2S assumed to be cheaper and the cost com D 
eompmceee whe) Phe Ccntract price is le 

eStimate, further adjus-menzs are required to ev 
d@ipact ctf th] two aitéernatives on the overall 


COV -Lateat: Opera fan T) = « 
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There are «we adjustments nade ¢t 
Pieet, -he cost of capital for assets that 
in-house performance must be added. Ssc m 
cost asscciated with a néw start where the function is rot 
presently performed in-house must also be included. 
The contract price must also be adjusted for several 
Romeo CLS. Acost of capital charge may be added if gcvern 
ment assetS are required tc assure contractor perfcrmance. 


SenverSoereccseS are added to reflec= che cne time ccsts 
m 


Peete eagmmoy the gcvernment in shiftzing c¢cpnerations rer 
Be q7ouse tO contract. WEeUrCOCAestCuLnd OUT WOUlG*. 2SSuLt in 
eeecadue~icn ce the present level of capacity, the additional 


amount of overhead which must te apsorbed by the remaining 
Bemavetses aS added to the contract cost. Finally, the 
potential federal inccme tax revenue which weuld be paid by 
MeammCChotscetor 1S déducted from his contract price. 

After all adjustments have been made, @an existing 
Miemieus=e E£UnCtTION is not converted unless the prejected 
GOicrace ELice is lcwer than the govertnment estimate Ey at 
Jeast 10 percent of the in-house srsornel related cost. 
doeeomds ete tens tal. ic includsd +0 account for th2 Pp 
Mescr Ce eLoguction, the ~emporary decrease in effi 
erfectiveness, and eon wnpredstetabil=. Sisks th 


Pom ou-sace COnversi:on. [Ref. 12] 


7. [ecisioa and Apo2el Precess 


Mereconobet ach Of the coOSt comparison, 4a recan— 


U 
Pere tenme ae fade tO Ert her award the contract cor to pericrm 
a 


Pio wEUune .-cues rn house. The recommendation, along with the 
CeowepcelcarisOn fOrmnmS are fompwazrded to the aprereving 
auc hneri< fer review and approval. Once approved, <he 


OV 

Eesults cf the cost study are announced and the @¢ 

analysis is made available «so the public. If no signifi 
W 


discrepancies ars identified or an appeal lodged 
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WomM@engescays {Which may Fre extandsaq by the contracting 
Geri1cer up +0 Vemmeays <£Oby complex décisions) =ha 
Seameactahg Ofitcer will aithsr award the contract or cance 
epe =FCll{ citation. [ieee e Sven ween> —CURCuzor is =c be 
performed in-houss, implementation cf the MEO must be initi- 
azced withir 30 days and be complet2d within one year. 
Peet. 15) 
Ds IBPLESENTATION OF A-76 

Cumecnt Lywoan estimated 400,000 fedaral aevernment 
emplcyees perform ccmmercial activities vomned. . aa ene 
Peeiecm se anhidily. Cf ‘this amoun«, Only, “oo uerliiorn are 


peergwbae £Or COSt studies; the other $14 billion werth are 
exempt from A-76 for teasons of national defense. RL = howgs 
Peegress 1S accelerating rapidly, +o date, only a small 
PemescnoeGt the eligible functions have received a ccst 

neThe Office of FPederal Procurement Policy (CFED) 
ates that a savings of over $5 biilion cculd be 
achieved cver the next five years if these cost studies wera 
Ref. 2] 


Wedwapprexidace.: yop 140° mitts cor 
year as a result of CA studies. In addition, an average of 
4,000 cerscnne BavVe Deen Conver=ed to contsace in 

L 


C 

£ h st fcur years. en oher 2) Daca oe. a 
Teidiecvemmeioc2, Snscowed shat o0 percent of the functic 
Meee d £6 Centract end ch2> avera 

ee. Tro cemerecuce sons Wet wicely ak oe 
Dem eeeymmeweta  cwOS-l2fthns showing greater chan 30 percent 
Savings, ancther two-fifths having savings betweer 11 and 25 


percent, while the remainder savei 10 percent or less. 
1 





In FY 82, the Navy hed its best year to date in imple- 


Memeengq the A~76 program. A total of 252 cost studies were 
Completed ; moze than twice the number for the crevicus 


tareée year period. Cf the 5,487 civilian man-years studied 


qt 


u 
BpvOO0 WEts contracted out resulting in &@ savings =o the 
c 


Meweornmert exceeding $15.9 million. The remaining functicns 
that were retained in-hcuse netted a cOSt avcicance 
eeeecding 317 tillion. [Ref. 16] 

Sect ts Hax=ch 1963, the Navy has identified 53,457 
perscennel billets as candidates for CA studisgs. mcs Oca. 
Bepoentacse Under NAVFAC's £unctional/contractual resccnsi- 


u 
Eeeaicyewas 34 pezrcenc. Although NAVFAC stiil has the highest 
percentage in the Navy, its share has decrease 

when it was 2s high as 70 percent. NAVFAC «ock n 
Memoir CA COnzracting for the Navy oy providing standazi- 
P2eG@mrertecemaice Werk Statements to its fiel v 

Cf all the NAVPAC functions studied, che €£ 


a 
meted, “Grounds Care, building maintenance, and 


waa Spcrta a 
PawecOrial Services comprise 82 pércent of th2 tctal. 
{Ref. 17] 


26 





DEL. CCNDRACT CONVERSEOMPIESSUES 


A. PACKAGING OF CA FUNCTIONS 


In implementing tke A~76 policy ther2 are several imper- 


memt COntzact conversion issues which must b= addressed. One 
Ss 


issue ccncerns the different methods in which Ccmmercial 
AGtivicies functions can be reviewed and cost compared. for 
Sees, a Single function from =he activity's inventcry , 
en as bus services, can bes Selle 6d ee Cay wedvin 2 
Alternatively, several functions may be combined into a 
package ratiors and main=enance. When 


Such as vehicle ope 
rn 


ieee £UNnC= Ons are consclidated by an acvivity, 


a tal Ease Operating Suppcre (BOS) package is fozrned. 

A multi-function approach offers many advantages cver 3 
Single function one. Lt Lacilievaetes smplesentation cf A~76 
Beeelscc 2c peduces ¢he number of cost comparisons. As a 


result, it is a very appealing option to these commands wao 
SeewunhG=er intense pressure to guickly cc 
Navy experience has shown that when smaller functions are 
Seow Gellrared Under .A-76 on a Single £ 

aze extremely ikely to remain in-house. However, whe: 
ep@ecemsame TUNnCtIOnS are consolidated intec a multi-function 
Paesacemmeney are €£a> more likely to be contracted cut. 


[Ref. 16] 


iemetciait== Cl SEr¥ice COntSaccting iS such that weanscal 
Diemmcrte =e S sisirtset= 22 TO SNnsurs setistactory perior- 
mance. What is needed is an 4¢f£fectiva management organiza- 
peemmrciec Can GS- ehe jcb done vroperiy. Pedic fire. ee. 
Serremaeccmegetmweces thas requirement end because of its 
Qresater dollar value, BeeeaGes harge: tfirtmS that aave 
sncreased management expertise. Mweechne CUuncouLlons (ere 


oy 





Getolek cr time ccrnstrained it may require the vast 
G=sources of the large: firn. Consolidates) OF Luncticns 
Gan else lead tc greeter economies c= scale and mere sft 


Client use of personnel and material. 
i 


Iss process of contract administration is nuch 


easier when deaiing with a singis larg2= contractor rataer 
than many Small cnes. The command does not have =9 contend 
meen Meny differing company policies, Peeceduces, and 


perscnnel. therte is a higher probabilic 


ith 
O 
| 
i) 


a 
eewoOrk because whenever a problem is identified, only 2 
Seige eGINt OF Contact need be reached. In addition, *hWere 
are fewer Spperecunitles L£OR. “COS Tractor 2 


meges-PCarting With multi-function contracts. 


Be SMALL BUSINESS CCNSIDERATIONS 


(eee une t 2 On packaging, while cf great value in impie- 
Deen geers Carcular A-76, comes in conflict with anctner 
feetona, Eclizcy; the Smail and Disadvantaged Business 
Meet eZecaden (SADEU) Frogranh. Puplve Law 95-507, Signed on 


Peo etcoer 1976, crrovades the legal ffamework for <=he SADBU 


Paeirante  scce2On 21 cl this law states: [{Ref. 187] 


It is the policy ct the United Statés that smali busi- 
Meoomeecenccras and smeli business concerns owned ard 
Cememeineq by Socially and econom-cally. ee sadvantaqed 
miegveadlwans, Shall have the Maximum plracticabie coportu- 
Meee cepate Cipa=te in the performance Gficoneracts let 
Dy any Federal Agancy. 

A sail business concern is one taat is independently 
cwhed and operated, and qualifies under guidelines estab- 
lished ae tho Small Business Administration (SBA) Wego 
regard tc number of employees and annual receipts. A small 


m 
dz2Sadvern-aged business iS cne cwned and operated sry a 
fmol iavyeteiteck, Hisfenzc, Americen Indian, 2*c.). In cracer 


a: 
Wem eoe toss COOGStUnNLtyY <O participate inh Jjcvernnent 


ip 
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contracts, certain classes of procurements have keen ser 
meee omrca ones CXCAUESIL VE DATticipation of smail business oz 
have been granted 8(a) set asides for small disadvantaged 
bkusinesses. 

Czreulacw A- 75 sequi Piercy COM CLAC eS Whaca Wav 
been awarded under authorized set-aside programs wiili net be 
BeWlewed fcr possibl¢ in-house perforaance. It also directs 
that furctions previously performed in-house thet are 
peace tO 4 Set aside program pe awarded without 4a ccst 
compariscn. On the cther hand, A-76 states in-hou 


eee excess Of 9100,000 arnually will act be sez aside 
a 


eect c GON VermslOn iS justified by a cost analysis. This 
last statement has been interpreted to allow unrestrictad 
pwr Geor mUlti=-furcticn contracts 2ven though the individual 


O 
functions, if awarded sacarately, would require being sat 


aside to small buSiness. 


On 16 March 1982, Congressman Joseph Addabpbdo ee} 
expressed his concern to Secretary of the Navy John Lehman, 


mma= ccnscladation of base suppor Services under BOS 


@ 


comecrac-> would devastate the Navy's SADBU TrOdr am. 

Congressman Addakbo claimed that many of the functions teing 

consclidated were traditionally performed by small business. 
g 


P-mud= tice Nedeemet because of the contract's lar 


complexity, stall Eusiness could no longer become frrine 
Pemeceetors, credazdiess of their prior performance. As 


[Hes pcemecmeca chaz large primes will “usuaily choose not 


tO sukcentract to small or small disadvantaged firms for a 


Cae ete co a veGo runccion.” [Ref. 19] The Cornrgressman 
cluded ty requesting t=h2 Navy stop all consolidations. 
PRomomecmetow—~ Vom teeter: dated 23 April 1982, Congzsesman 
Bedabko —~chaged Secretary Lehman for not sending hina a 
substantive reply te his criginal frequeées He also charged 
Ride eee vy nad accelerated its errorts 20 exclude smait 


Zo 








Wwemere ty DUS Mess from Service coOnm#eaces. e@As eve icemee, 
Poe C2e-d the $6.9 rillion”™ BOS centtact awarded at Neval 
Weapons Center, China Lake, California on 18 April 1982. de 
meso Sedientitied six “other BOS ‘Solicitations that were 
recently released by other Naval activities. I n@elossng, 
Congressman Addasbo renewed his request tc halt all further 

Bemociidaticns of CA functicns. [Ref. 20] 

In hig respons¢, Secretary Lehman stated, {Ref. 21] 


fem comer clOnd= pe lief chat Gensolidated contracting can 
BomeceCOs= ottective sttateqy  cften fostering 42 acze 
efficiert use cf scarce acquisition B2csOugess. doweye:, 
I. aisc pbélieve that consolidations must be undertaken 
With a keen awareness of the objectives of the smali 
business program. 


Secretary Léhman maintained that small business has hai 
@Pedotacratle success in capturing multi-function contracts. 
Maeche China Lake award, he explains = Nai cosa Cl Ome eS 
meeceeen fLunetions, representing 50 percent: of the co 
value, were subcontracted to small business. Pa aya eS 
PoeGetany ELoOmised that a Signitican= portion of tfucure 
Bemsciigated Contracts would be awardea@ <9 SADBU f22ms 
piven, feral SE in part via Subcontracting. 

On 24 May 1982, Secretary Lehman sent a memorandum to 
the Chief of Naval Oferations (CNO) outlining che Navy's new 
Peepeyecn consclidated service contracting. it makes *hrsee 


major ee Coal 22] 


elee@encolidatio Ger =xXisting small business c¢cntracts 
Shall cnly be considered wher there is 2 reasonalle 


Maeeereac Ot. acme "Weld resul= in an award tc snali 


business. hieenen es veme | that such @am award is nox 
accemplished, Mae v Natale sUNCTION COn=Sacctang mus= de 


=ssinstated. 


DyeaunGeachs whlch are currently ing performed under 


O 
J 


ClicieGeacracts Watimmoec De considered tor consclidaza 


Umwese consented tc by the SBA. 
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Symeeomcracctrng OWL OL CULTENt in-house CAmfunctacns shall 
be done giving the SADBU Program careful consideration 
early in the frlanrning stages. In thos instances 
MimetS SMa@ii business participation is determined not 
Semcon Ue cone best intergst of the Navy, supecccsang 
Justification must be forwarded to CNO (OP 443) raor 


ROmcQlic tat On. 


Cne week later, cnt June 1982, De 
Peecnsewraerk Carlueci promulgated ths 
di. 


» 
ieeene CLirect today. The oO0lgewmed:sectSd cheat any Zume- 


Seal. 
tions currently perfcrmed by small business shail nsct be 
eem@emiderced ster COnSOlidacsion. In addition, the Carlucci meno 
swaczed that : [ Ref. 23] 
Miewreees Sclicitaticns, uniess meas al 2. OVSsr. ding 
Poiomabeseci=c ity considSlations, wiil be eae sc as 
PeewmsG  ereclude performance by small and smali disadvan- 
eagiea GCCNCerasS aS prime contractors. 


Although the issue appv2ared to be resolved, it has again 
Sitemeeced Wath =2e Cfitice cf Feieral Precurement Fclicy 
E@obieeceag ESvVisior tc A-76 that is currently bei 
Pa (eee crOpOsal Would direct that Consideration be given 
BemeceCdseledeting CA tuncticns into a sirgle stater 
emeeece GCOst COMDarIsSOon aid potential contrect. while 
Bateweeongessnat Cconsclidation may reduce prime centrpact 
Sepeo twig tes -Of SALEU concerns, it only dixects a =reascn- 
able ralance be maintained between consolidations and single 
Pees OGmewaeds <~O Stall business. ({R2f. 15] 

Pmmeatdeeton 96o CFPP's tecen= support of multi-function 
Backegisg, <h=> new Detucy Secretary of Defenses, Paul Thayer 
haS Stated that he intends to change che DOD consolidation 


a 
Pereyra ee OV has pred¢ssor, Frank Carlucci. Reccgnizging 





Peete ere cuctenc policy ell but crecludes gulti-funczion CA 
contracts, Thayer favezrs an approach which encourages ccnso- 
lidaticns geared femme as gs £2cns while simulta 
MeeveG=ng Succontzacts <9 SADBU concerns. [Ref. 2] Eoth ch 
GaeP Levisacn =O A-76 and the néw DOD policy or consclida- 
tion are expected to ke precmulgated within the next sever 
Months [Ref. 24}. 

Arprceximately 300C Navy CA functions invel 
10 civilian trerscnnal man-ye2arts. Even when co 
PEt a=tUnCcLOn cCcon=racts, wany will s73 


Ten Caperbslity or small businesses, Of the 252 cost studics 


conducted in FY 82, twenty were greater than fifty man-yeaérs 
and cf these, only five wers over on2 hundred man-yéars of 
cr ort. fieeer. 16] Algheugiho65 percenzse o ere 


testricted to small business, these se 
few2r ccntract awards. Ahi beas > DEL Cer: O Leechs 


Man-years in the unrestricted solicitations wer2 ccontrected 


out, only 37 percent of the smali business set asides were 
awarded. [Ref. 25] These results clearly damonstraze that 
the larger unrestricted solicitations are more competitive 
meee one gGeve-nment and lead <0 greater conversions tc 


Cane Tact. 
It weuld appear that the Navy cannot achiev 
age wrameanes sought Ey OFPP unless it is alicwedi gre 
E 


Peace onmeconSclideting CA functions. WS additionéel fuinc- 
i 


mono meaee sub ;yecced =O CA cost studies, an nereasing 
BamecueacemWwell be mere complex, Critical, ane STf2icienc 
Since the easile Spoetess Clr icien= =L£unc 


Pome Oimmeca=¢d Olt. Lo LS therefore pradictable th 3k 
dual fuancticns and «hose restricted to small business will 
e 


have ¢ver less success than at present in winning ac 


Q Oo 
SJ 
: cf 
V4 
fas 


award. Thus 2 total reliance on single ‘function pack 
Die eeveeemenly De d=trinental to small business but will al 


Ss 
severely handicap the a-76 aim for economy and productivity. 
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C. QUALITY ASSURANCE OF CA CONTRACTS 


When the federal government purchases items or services 
Mnder a contract, it assumes the responsibility for ensuring 
mee SUCH 282emsS OL Services ccniorm to stated contractual 
requirements. Two ccncurrent processes commence at contract 
award which influence satisfactory qualizy. 

peommGencsaccOr establisnes .a quality con=rol prcqran 
wherecy Maragement cecntrol cf materials or services is exer- 
Sesead cor the purposs of prevention of defects. In govern- 


meat Contracts, SG= CobembaGeOn asslmes Gs sssocnsibilic; 


5 


mee, ehe CXecutlon Of the quality control process. [ Ref. 26] 
Pecne same ¢imle », 4A gOveramen= contzsact inspection 
Seganiizaticr administers Ae ia ee assuzance eperocess. 


nce isa planned and systematic approach of 


= 
cbs¢erving s¢rvics performance to provide adequate cerrfidence 
Ss 


jade tne LteMS OL S@Evices conform 20 EStabiished technical 
requirements. The quality assurances process veriries the 
required gquelity of delivere LtamS OF sSAarVices prior <tc 
boeiit accepzance. 

tieoet  Cibment A=-J/6pecpclicy, twO ke documents are 


G 
Be-Oae-d dissng <he gcre-solicivation phase that  iniivenec 
GWeessey COLt=Ol and vassuraice processes. Ihese 


u 
ar= known aS a statement cf werk end quality assurance cl 
1 


ate 
The erfcrmance-oriented statement or work describes the 
Mea. mus cequizesea ievel of services that wila be exoected of 
Pro ecMececcet 'Gem-t aGzO>. The qualiizy assurance pian states 
ve Beceaules meecnacemwill 5e used to check and yve¢riz} 
Beet ccec= Pol tCrmance. 

Pe eemecc cre 1679 =Siyision of <aA-76, Solicit =e2acns 
Bee ieoc  Temiuseaq design specificetions which e¢setaeblisize3 
Precise RBeasuremen=s, tolerances, QemeCiakity YOCheso. 
Bequ@eereM<owe EO ene centsactor. Other detailed informe-iocn 
was provided when deemed necessary. These specifications 


a 





feme pemmeraly intended to obtain standandization cof delav- 
Seed 2e-oms Of Seevices, In CA sezovice contracting, such 
Specifications often proved to be unwieldy and ineffective. 
Standardization was net always a suitable objective for the 
service contract rrocess given a wide Ga Ve Boao ie 


eontractczr skilis and the Character of operations “at 

ifferent Navy activities. The government assumed substean- 
eeieeb cok With) this “ethod in Wat it "guaranteed acceptable 
tesults as leng as the svecifications were followed. 

Gmeze wenlery 1982," an OMB mnemorandun ditectéd that the 
Gerace Cn Pederal Frocuremen= Policy Pamphlet No. 4 be 
désigqnazed as Supplement No. 2 to Circuler A-76. I* Further 
required that both work statements and quality assurance 
BeemestO= CA functicns be written in accordance with «his 
néw supplement . {Ref. 27] The OFPP Pamphlet embraces the 
Wem@esprecdd utilizaticn of performance-oriented work sta 
Benes instead cf detailed, exacting design specificaticnas. 
Mcmmmccierde ot 12S cisarly and precisely tela what is 
required rut not how he must do it. This allows a ccntractor 


meme ieexabilivy iin perforaing th2 WOrkK. An objectively 
Soeeneae Nd MlOodwect Eacilizaces che contractor's quality 
Geierch and the govetament’<s quality assurance effort. ane 
gualit assurance plén gives the government inspectcr a 


detailed, Meittcen Elawe which allows him to accurately 


na 
moseSS ne cecn=ractor"s per formance 
$e 


The statement of work design encunners several review 
ave -eanpemesec=essces., First, LUNcezional areas Subject tec ccst 
Pld y MiUeemrecu@omple-ely d2fined. All rescunes inputs, werk 
aeG =o cect DEOduC. TON OUTPUTS Feguired for successful 
pen se ne ana gnee- 

3 


EmNance off the jck function must be ids ed 
De Curs ine which 


die After this, a job analysis 

ure of the organization is de 

eseetgmeteweox -l=mercs £Or the function is enumerarced, a 
an 


n 
Standards of performance for each work element are set. In 
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Beige ach, Lesources tc accomplish work input are dezera 


- 
* 


3 a 
lee a 


ay 


e 


Pemocenmamce “NdicatoOrs are listed, and deviation from accep- 
table standards are specified. reece 2o ie When the jon 


analysis is coOmpiste, the essential rudiments of a contzrac- 


4 


eieal Statement of work will be formulated. FraniGt 1 Ola a= 


(iD 
~) 


managers then ccensult with contract specialists and indus- 
trial engineérs +o ccmpléte the contract package 


The Air Force and Navy aave promulgated reguiaticns 


which implement the precepts of OFPP No. 4. Over the past 
thr2¢ years, beth have issued standardized statements of 


& 
work (SOW's) or pertcrmance work statenents (PwS's 
Bemcaam tO Specific CA functions such as refuse col 
grounds maintenance, or any cther areas list2d in aA-76 
Sevyentcries. hee le lGtaGts vity perterming A-76 cos= studiss 
poets ze these PWS's and tailior them to iaccrperate the 
Epecidgmelccal Teevirements. Approximately twenty ePwS's havea 
Beeaw WE2tten £02 Navy Public Works commercial functicns. 
Standardized quality assurance plans have also 


D 
prepared for each statement cf work and can be tailored by 


local activities. These state the methods that will be used 
ma UNS Dect ing all contract eee S The Naval 
Facilities Engineering Command ids ies five methods of 


Seeveliliance: f Ref. 28] 

a) One hundred percent surveillance involves insvection 
Ser=acnmGicecurence Of contract cutput. It 1S expensive, 
time consuming, and not 100 percent reliable. 

b) Planned sampling allows part of the contract output co 


be evaluated. cova 


a 


Sim jege:vs,s and gerezally us¢erul 
@iemanet. Gertain items Of work are very i.moortant. 


Cc) Randcm sampling uses statistical techniques to samecls 


amepeGewsom Ch CGNtract OUtTpuUtS. Each itsm has an #qual 
Chance of being avaluated, eliminating inspection 
Peas, 
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an 





yepervccncduicad inspections are imeoromptu spot ckecks 


O 
ih 


PHcmecnhebacoOor’S Pert ormance. 
e) Validated complaints re2sult when customers express 
dissatisfaction with contractor performance. Although 
ccuplaints may rot be used in lieu of the ther 
methods, they can be used tt 


O° S 
inspection methods, and make payment deductions. 


The method c£ inspection is tempered by the various 
fyoc=S and frequercies of work. Repst 
mance may Fe o¢st suited for planned a: C 
mec~hcds while intrequent, critical work items may req 
100 percent surveillance . Once a method is chosen, inspsec 
Megiteesecheaules are created for each month of contract 
Pestounanes . EValuation worksheets listing each werk item 
are prepared, inspections are conducted, and results are 
BeeGo2zded. Good performance 1s teward2d while poor perfer- 


Gance 


(a) 


required to be corrected. 


t- 


Wit=me Gontract Ssclicitation is issued, the statement of 
work and other mandatcry (bcilerplateé) provisions a 

meea sand Gictzcibuted to prospective bidders. Rie Ve. Pouce 
Meee dcpted “he practice of distributing its 

rance plans (less actual inspecction schedules) 79 €ach 
meadi@er In am azeempt co alert them of the contszact's quality 
assurance standards . One kéy requirement of this entire 
Paewcooe sas Claticvy, precis2 wording, lack cf ambiguity, ane 
conciseness. petememd tecaunical NWOrding .tends tc be 


confusing and must be avoided. 


The authers observed that in Several instances, mors 
Pee emer Guee= given tO preparacicn of =he statements of werk 
Scr OurGdtci:<y assStrence plans. This is understendaole, 


a 
Since many DOD activities are rushing to comply with th 
Bev omccduwuemens FO Ccnplet]e cost studies by FY 84. In spite 


Pepe meee Te CON=trcaint, it would b= counterproductivse for 


30 








an activity to mcve tcwards implementation of widespread CA 
Pence COliwmacting without a specific and defimeciz a 
fOr guaiity assuztance. 

eee xt Gkapte=S GiSeCUuSS Gniciatives in <sffiectiv 
E@meverliarce Bethods and properly stzuctured Aanspecctio 
organizations. A well designed quality assuranc2 oprcgian 
will cesult in the optimun Use oe. Scasce “Enspeczio 
resources and will help ensures adequate pezrermance by C 


Semysce CCnhttactcrs. 
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IV. STATISTICAL QUALITY ASSURANCE TECHNIQUES FOR A-76 
CONTRACTS 

A. INTRCLUCTION 

During A-76 cost studies, three documents ar¢é rrerpared 
Mei play a Key rele in the Commercial Activazies (CA) 
Seem erec= ELocess. TPE Bir st ifs a Contzactual stazenen*t of 
work which defines minimum requirements for services ordered 
Memenie GOucudec., the second, 4 quality assurance plan -wnich 
is designed and integrated wit the statement £. Wee, 


u u 
n h t 
Summarizes those schedules and techniques thar 
a O° 


essantial document, the cost ¢stimats, reflects tne acs 
ezfective and efficient government ennoloyee pertcrmance of 
tae p@rceicrn being studied. 

Cnée prime okjective of A-76 is cost effective mission 
perrormarnce by federal agencies. This is demonstrated by 
GOompating the ccst estimate cf government employee perfer- 
manc ian a 


PES GIEGIGeseGnaercd by COmMmpSsting cOLtractors whe 15 
Capace @0 A=#6 firm cid procedures. Tae low offer 
Pima e s(youlaliy) <cC Dericin the tunction. 


MieseeNenasrs Of COSt E€ELSCtTive verrtormance cf govern- 
h 


Beant operations Noms lectsed in ehet suadeance “which 
Seas ecMo ee CeScatarazcn of A-7/6 CA quality éessuf®ance 
plans. (CCD publications describing <the d3asign anc impiémen- 
Pore @ncumdual cy aSSubance programs arte : 

Dene betee Regulation 400-28 of Sevtember 1979; 

t) Office of Federal Procurement Policy Pampnlet Number 4 


Gerece.ouer 1980; and 


Q 


Naval Facilities Engineering Command Manual MC-327 of 


Nevember 1982. 
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Sdet@epor tnese publications cails for dévelopmen: of 
cost-effective quality assurance programs ky e¢ach fiel 


activity te provide reliable assessment of servics contract 
Eerfommance. Cest-effective surveillance does connote an 
inspecticn system being created at the leaszt possible ccst; 


more afrrepriately, the important facet of such quality 


assuranc? is to utilize existing cor new inspection systems 
and rescurces ina reliable, dynamic fashion to gain tne 
Pest detertManation cf, contractor performance under CA 


contracts. It should have the capability 


ct 
O 
i 
(p 
ip 
pu 
W) 
{4 
}~ 

<S 
3 
O 
fis 
}- 

r 


fied wher CA contract worklcads increase. 

The Naval Facilities Engineering Command asserts the 
impertance cf properly designed quality assurance prograns, 
contrasting old and néw inspection plan design philosophies 
(Ref. 28) 


The  Navy"s traditional ap 
service certtacts, or zen an 
Wetec plan, has not provi 
ma GCs ae The method of survel 
<o Fe used Gee freguently is 100 
real: however, such inspectio 
1 @hs Fecticn Since SGuLs Very c 
meet oer, «rad. tional surveil 
cussa on =A WOLK process (4 
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id not always 
methods have 
NGS, 20 =speci— 
Thea ual tay 
rot assuré 
y assurance 
quality: Buisea On a write: plan tO (DesLer= 
mance oriented spec: sficat ions. ters the gqualaty 
Seer demecneaquc=. delivered by the contractor and nct on 
the steps taken cr rocedures used £0 provide @ havc 
il Gelene vie bcos Tt ircludes agepropriate use of preplanned 
inspections, moe aeon Gf: lcuscOu=5.complaiacs, an: 
unscheduled inspections. Le DaOVSCes 6a Seruiceuced 
ars Oach 0 Survelijance that permits management 
Ceveuue Ce Ggudlicy assurance. 
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SeOn addMiniseers quaLiic CEnmrolns OMece “Det tor- 


Ta 

mance *o ensure that a minimum level of s¢rv 
Quality 15 top management's r3q 
we 


Q 
& Which stress that these be vrovided ata 
e m 


a9 





te procedures of tcp management used tc determine <+ha- 
eigsce ACtivities are being carzise @ut in &@ manner zhat 
WaS established by prior planning and goal setting. 

Quality control is aimed at the prevention cf unsatis- 
factory performance c£ CA contract servicés. COU= =a SE20r 
pregrams fccous on developing employee self notivaticn «to 
render acceptable performance of servics. Cislity Coresot 
Meagoeco tncheased profits f95 ths contractor and haga 
Mewele= c= custcm=sr sa@tisfection, ane fect tates 2nelcased 
empLOQye> Ereductivity in work performance. [Ref. 29] 


Gsuecilase A-76 emcrhasizes effective contfactor gualit: 


Q 


onzrel ard government quality assurance by embracing a new 


myoeee OL Sunve:liance procedure known as random sampling 


Biesececercr. Traditicnal planned inspection and 100 percent 
inspecticn systems are sing replaced py these having 2 
tatistical basis. Statistical sampling techniques increase 


the objectivity of gcvernment quality assurance sin 


c 
Z2Sem cf service has ar equal chanc2 cf being inspected. the 


numbér of government inspectors régquired for «he surveili- 
DeanGe ~~ plLecess is reduced, resuiting inieéess laspection 
Gosx S . 


E. THE EVOLUTION OF STATISTICAL QUALITY CONTROL 


Meese n ce mex amunatton of She cere ane ea On 2onag uh. =*c2v. of 
meee ee Gel Gual2ty ccht=ci will be de 


Ss 
Papeeaeesa-28Gel Quality Gcntrol (SOC) éolic 


eee eo a fewi.Y COn=cOL 15 the endloyment of statis- 
SeeclmeemerctolSS and methods which have been develoced 
so assess the magnitude of ‘chance cause variaticn' wand 
Pemeg- 2 ekemetasS-gnable cause variaztion.' Variation due te 
‘Chance causes! is inevitable while variation ee rae 
‘assignable causes’ sane Usually pe derect2d and 
corrected oy appropriate methods. St atist neat quality 
Soe eeet iene hesecpny 15. the early detscticn of assi gnakle 
causes so that product, quality may be “Gontroliea at the 
Gesired level with a minimum of rejects. 


4 Q 





Assignakie causes of variation are due to differences in 
machines, wcecrkers, materials, and either their cemposition 
oz réelaticnship to ¢ach other over time [Ref. 31]. 

Seem pecocesses CCntribnute £0 economical achievements of 
Peoduet “qua ] 3+ Dy applying a body of theory dealing wit 


laws cf large numbers and probabilities to various indus- 


trial and service prcecesses. These originated in 1654 when 
Pescal, a trench philospher and nathematician, teamed up 
With Fierre Fermat to develop the theory of ~obability. 
{Ref. 30} 


Untiz? che 1920"%s, most Semblances 9£ SQC wers asscciazed 
with measures of central tendency, or averages. Increasing 
study at this time was devoted to che eifect cf standard 
déviaticns cn control precesses, and led to the foundaticns 
memiea@e@Meseats Stical quality control. [Ref. 31] 


Walter Shewhart cf Bell Telephone Laboratories develored 


fMe-ecaeet soc model, known aS the guaii*ty control chart. It 
WwaS used to measure product quality vériations by both the 
Bell system and by Western Electric Company, and was 


= 


augmented by statistical saapling techniques of H. F. Dodge 
emis Gs Rem. Gg. [Ref. 32] 

Hedeserecd Svolution cf SCG “semis cae con wetechnagues 
Seens-d) a worid Wa= Ii due to “ne need =o Rinimize preduc- 
j.On scrap lesses. Government agencies developed training 
courses for these fersonnel in aniustry who inspected 


product or s 


iD 


Everce (CUcDUtS. Sid Waz if, American 
u 


A O 

Benere= developed SQC *2ecaniques, waich were 
édopted in Europsan and Asian countriezs as well. S9QC became 
an — basis ‘for many Bodtscesiate OS OGUC avs 
till utilized excensively, as ¢tvidenced 

in modern Japanese industries. [Ref. 32] 
ion may lead one to believe that SQC 
sechniques were designed primazily for manufacturing 2ppl:- 

s 


pewOUL these are caquaily valid for non-manufac* uring 


uy 








ey@es “Of technisgues. The following instances illus«raze 
Bepeocemocetve applications: [Ref. 31} 
a) SGC Was used by the Census Bureau to control clerical 


accuracy; 

b) Aldéens Inc., a mail order business, util 
Sxteeoeto e€Stablish accilnacy in customer billings; 

c) Tke Iliz:nois Eell Telephone Ccmpany used SQC zechni- 
@mlese <> assess clerical accuracy in the acccun 
Geras tment ; 

@) The Standard Register Company used samoling flazns to 
GeEQeae. accugacy of sales invodcss; 

e) United Air Lines used control charts «o improve accu- 
racy in customer bookings; 

£) Statistical control té¢chniques have been used success- 
miei whe health cndustry fRet. 33] 3 and 


Syocartist2 cal guality ccntrol techniques have been used 


et highway and alrpore pavement SONSESUG= Lon 
{Ref. 34]. 
Based on the preceding observations, ie iS) Seasepeadle xo 
apply statictical sampling techniques cto CA service contract 
Surveillance programs. Such tecnnigues have been utilized 
bye <he Air comics Wm its A-76 Contract cConversi#emc. 
Beoceqm@eecS 2Or arolyim@e these technigues are state in Ais 


rorce Regulation AFR 400-28. 

Beis Zegimaeton Gacorporeates a statis 
mod21 knewn as acceptance sampling by attributes. This ao 
momere. fugy described in Déepazcment or 2D 
Beem Ce 26 Acra il 1963 (MIL-ST9-105D). the stander 
has been used successfully by defense industries since its 


Sertoi7c! tcmlilatmon, and its concepts form the basis for 


worldwide acceptance sampling standards. MIL“STD-105D can 
omer aetc ermspece® tre following: [Ret. 35] 
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eyO cela «ciOns; 

qd) Materials in precess; 

¢) Supplies in stctage 

£) Maintenance opera —. . 
Dewan ce =6' COL dS; and 


) 
h) Administrative rrocedures 


Mees Lis) iS act exhaustive. Rho: occ OL MeL -57 9-195) eis 
presented in Afpoendix 8B. Racscq eee r condi tica2e for 
BeleGesS;Ul use c= SC¢C chrigues are tnat operations be 
ss) pace 


ependéent, fuactional, and affected py as fey 
= S 


ecscupie (Ref. 3ije CA sezvice..c 


A review of representative SQC literature revealed the 
EoloOwanmquorbjectives: [ Ref. 31] 
a) It indicates the poresence of assignable causes of 
variation; 
Demetersdt=s c= Specific sources or these causes; 
c) Iz is as simple as possibie; 


i 
d) It leads to remeta chances of searching for assignable 


causes of variation when these are rot vresent; 
Peete menS COStS, =sducing Lapor and Matstlais wast; 
=) Lt smproves quality, making such imprevements 
uniformiy throughout Temas PSOaducr = On OL = 2fVice 


Bae Geos: 
g) It sets and ajijusts tolerances and specification ktased 
nd 


h) It improves employes morale ate the Sener of 


Snmecculred DECcduction Sxperliencs; 


m 


GUseeter-Vvendose relationships. 
iMiG@oEre banlon or SOC £echnigues in CA service contract 
administraticn may initially appear conrusi G 
Maso ne tgewneen pPpLeSencts 1t in a cisar, concise manner sho 
oC: “his apprehension and Jéad 
<echnigues py bcth gevernment and contractor ir ae ass 
ED 


Tance psrscnnéel. Generepcssi bis 


4 3 





Beeunrgqucs May ot Ee appropriate for cartain business or 


and managers may mistaKxenly belive 


services would always be performed in an axcesllient 
c£ SOC me 


meen Object icons m@ignt bse minimized if 


pa een. 


exclusive of the use thodelogy. 


“ne adva 


Bee, ere presented. iayssduce= Scaaemecn@esewer kK, inc 
quality awareness in all employsss, and enhances ozrcduc- 
aoa TY. Bheseicads «tO incrsased quentit@es of inproved 


rroducts and services. 
Beleatli= so that 


CcCUr. 


bi SPeCtuLonmmoleones Noes seven t. fic 


ard redictionme: wWihoending cTrourle cam 


Inspection costs are reduced, while authentic and 


accurate records of quality can be created. 


oe LEGALITY OF STATISTICAL QUALITY ASSURANCE IN GOVERNMENT 
PROCUSEMENT 
1. Intzeduc*ion 
feewore many Geel Of government procurement is to 


obtain timely and acceptable delivery of specified sezrvices. 

Memesactcrs are alerted to tchis gcoai when the gcwernment 

includes mandatcry clauses which St aero | tesa > 3On- Seeen 

G@emaucting gqualicy assurance. A typical claus2 from Standard 

eee z5-A (Genstructicn Contracts) is as follows: [Ref. 36} 
Gee ork (hich incinudes bucwes not tes sicted =O fater- 
ialis VOmmMana=mo, and Manutaccuse: anc cabrication of 
component s) shail ‘be. sllvecCce OO LNSOeetlon and. <Eest by 
che Boy a eae. ate all Seasonabie tim=s and all fFlacées 
eee wacee DeanCe. AY Such inspecl on and test is fcr 
the sci¢ bene ie of the SCE nasa: and shail not peliievse 
ce - Cee cCROT 6CIl UeLme espensibilmey of Cee 
Modena y Geloscl measures "£0 Sccite- Ciao Meenc ser 
See CelvecchiOL2eS Seach tne con=>act requirements. Nec 
~ASEECt on Smee St uy the goverament shali ps, construed 
eeeCence cl sue NGgmor snplylag oe 

Potecaoraily, O-her Special inspect aon clauses may Ampirty 


unique guality 


Eequizrements cf a contr 
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The government can exercise great latitude in 
Pepe ct?cn cf a contractor's performance, conductir 
reasonable manner but incr¢asing its ree tiecig pea 
cant cefects in perfcrmance are datected. Unspecified means 
of testing are allowable as long as these are reascnabls 
Senmaucting surveillance. Methods shat increase contractual 


performance should net be utilized; concurrently, governmen 


a 


surveillance should net interfere with contractor elfer= 
Mance. iInspecticn by means of sampiing prec2dures has been 
upheld in s8veral cases présanzed to ‘the Armed Services 
Peard cf Contract Appeals (ASBCA). [Reaf. 36] 

In any insp2ction system, th3 government must aveid 
Myer nrask cf effecting constructive changes <0 the contrecs 
by unreascnably elevating its own survsillanc2 réquizement 


Smrdidility contre] requirements for the contractor. 


2. Satpiing Techniques Sudstantiatsd by Beards of 
Cconzract Appeals 


SCC techniques have been sustained by the federal 
MBemets and by federal agency scards of Contracts Appe 
evo -fal Inc. , ReErGny 160720, a eee hy eae. fa 
MIL-SID-105D was utilized ¢c properly estimate qua 
defective ccats that cccurred in several producticn lot 


Government inspectors properly organized random samples to 
ec 


meee rcaelf Cont racto: performance and were COCLece tr 
concluding that sample results were representative cf entire 


Beeavcticn icts. 

n Goldring Packing Compary, ASBeA 77356; ISoZ2e sek, 
pata. 3392, a government decision to terminate for default 
on the kasis of sampling results was sustained by the Bcard. 
mene cked 91 meat loins ous 114 total icinas anc 


© 
round defects in each sampls. 
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The government's choic2® of acceptable defect ievels 
was Sustained in Precision Products, ASBCA 14284, 70-2 BCA, 
para. 98447 in regards to sampling inspection. I 
clear that such choices must ke based on reasonabl 
femme  seguiremerts; if defects are critical in nature, it is 

allowable to state that no defects or defective verformancs 
gee «CES CPermitted. 

With no methcd of inspection specified in he 
GOm-leacs, the gcvsrrment preceedei to 
ques in assessing product characteris qd 
eeeegeAc SCA 10259, 65-2 BCA, para. 5102. The ecard ‘found 
that sample sizes utilized wer2 sufficient to allcw a termi- 
naticn fcr default. 

ie cSOCUace Gieitcraect Tee) and MNamibtact urineeine,, 
Aaecn (255, 1963 BCA, the Board stated thac 


ce the government UIP CES Eo ce jcem anmcncicre jlot cs 
teas cr the basis of ah inspectioh less than the fuli 
quantity Gelivered (the. inspection Sampling), fhe 
laspection sample must either be representative of the 
entire lot, or in accordance with a Sampling and ccnzrol 
Beat acreed to in the contract. 
Tee beard also emphasized that inspectors properly désigqnate 


mi-wianner cf forming inspection iots, tnaé ee = 
Semple Sizes, and the manner in which contractors present 
Mots Cas inspection. 

Samei2ng ProOcedur=s EOCrTinspection of manutectured 
Beecuc comme == 2ncluaged itn Contract provisions for Metal Tech 
mc, , SoeGnelsco2d, 72-2, para. 9545. The Board sustained 


the manner in which délivered items were rejected and 


eieeac'a-c e=en= GOvernmMen="s termination of the ccnttact, 
@eeendgetne Frank and Natren decision which stated that “the 


SeveLoM=5t 2S NnSt Orligated to inspect 2aco detec: in all 


_ 


fetmaver=so eWepliie= when fcrming a basis for lot tejecticn. 
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in. a final case, American Quaiity A 


nh 
A 


ae 


nc 
Engineering, ASBCA Nec. 11417, 11466, 11544, ard 11747, 6281 


a) 


iT 


ECA, para. 6986, the government used a liquidated damages 


Clause which stated: 


The government will inspect approximately 10 percen= of 
Pen cesks pettosmed daily by the contractor. Of these an 
acceptarle quality level of a daily averagé cf net acre 
Shar sighteer tasks has been estadlished, If ‘fcr any 


Calendar month the contractor exceeds this average né 
Shall fay to the gcevernmert as liquidated damages for 
SrecesSs adiinistrative costs the stm of $250.00. The 
Memeey paverag> Of wnsatisractozry voerrorned tasks is 
Swee ned vy mov din mS voce ee ot PUuNsatisfacto-mly 
Memecuted *¢Sxks f£0r the morth by th aumber of working 
Gave for the month. 


his case, government payment withhcldings were ina 
dispute. The government based these on judgmental samplings 
r 


andom). Some withhcldings were sustained while cthers 


Were denied by the Ecard. Deductions from total nontnly 
invoices rased on defect percentages observed in the sampla 
sa 


fu 
4 
 & 
i? 
; 

ve) 


Memto be Correct. The Board di reed with enl 
2 ¢ 


A 
Gaon = 


rhs 


u 
gmrecte-Ntages to 100 percen= of the project 


d 
Syumeana Geplying ~aSse £0  =he SNtirte month's perfcr- 


Mance; it beizeved that this enlargement was conjectural and 
Peexecd SUutcae ent accuracy. Ue On Orese tl Std 9105) 
Sampling technigues might have injeccred sufficient accuracy 


n 
Memeo cis Sutveilliance process, leaving no room for decukts 
Rich eeconic.y Cf deducticns. 


SMe Om oo s-antiate on Of  Seaciszical Qualicy Assurance 


a 2 lab a SS ff - bapa =a = at — ap Se ee ae oe eee ee i =P op aS Se ee 


MeeLlecem -sGe%cra.  Aceccuncing Ofrtice (GAO) Sa 
Substantiates government inspection by statistical sampling 
He~wncds. Er Envitcrmental Aseptic Services Administration 
Seomlatectsluriding Gare, B-207771 of 28 February 1983, GAO 
evaluated protests against Air Fercs implementation of 


pees aperaecsd deducticrs using AIL-STD-1T05D. An issu¢ of 
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Meteo Was the defirition cf work items, as each one might 


GEcnSist of several sutsidiary tasks. For example, clea 


aug 
cf rcecems cculd involve sweeping floors, emptying *rasn cans, 
duscing tlinds, and several other «asks. Becauss of the 


moaweG -2Ck Provisions cf an Air Force CA contract solicita- 
tion, che protestors claimed that one defective subsidiary 
mesk could lead to rejection of the entire room just as if 
ali subsidiery tasks were judged to ba unsatisfactory. 

The GAO agreed, finding that the governmen= reaquire- 


f=] Ot tase Cs L=ascOnebls ara could f= Vviewe 


tu 
fv 
7?) 
pv 


Ss W 

Penalty system. GAO recommended that the Air Force distin- 
guish between vital and non-vitai tasks, establishing 
Beasonabie deduction rates for both. This nas resulted in 
more detailed breakotts of work ragquiremencts fer certain Alr 
meeece CA Contrac: solicitatiors. 

The GAO emphasized that enclosing thea quality assu- 
memec PplLoOvisions in solicitations w#as clearly for the 
Bem=tiz cr the gcverrment and not potential offerors. These 
Gap not be disputed, nor can failure by the governmert to 


meme te =-C them fcrm @ basis for pretsst. 





The preceding discussion points out a Sample of 


legal cases in which government use cE statistical guality 
assurance tecnniques were en a Tnese were fourd tc d¢ 
Walid and applicable to government procure 

G 


NMigues must be reasonable and notin 


CI performance. Jse of SQC techniques are ta 
BERemcGVerrh=nr.e and net the contractor. Rejections or deduc- 
zions based on sampling must te representative of the lots 


Sievers Cao SeiV¥ace being performed. 


SeemeecHiieauccunave, a legal besis wnhen aoplied to CA 
Semce Gontrac<ing. Psu eng discussion will e¢xamings 


{2s 
O 
i | 
$v 
t+ 
(D 


c 
Metere does ctasis+ Cal fethcds which ere being use 


O 
{U 
ct 
} Je 
O 
= 
6 


Mem—nGectua:=¢ £O> theiz pessibls eopli 
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D. AIR FORCE CA CONTRACTING UNDER AFR 400-28 AND 
Bri-siD-105D 


1. Introduction 


The Aix Force first issued policy requiring the 
concurrent design of performance-orisnted statements of work 
and matching qian 2% escieancey Clans “seq CA service 
contracts in Air Force Regulation AFR 400-28. Subsequently, 
standardized statements of work and quality assurance rlans 
which could b2 tailcred to local command needs were devel- 
coed and distributed. AFR 400-28 required the use of random 


Eemplang Erccedures based on MIL-STD-105D, along with cther 
spection methods. Other executive agencies follcwed the 


at 
femeercoce lead. The Office cf Federal Procurement Pclicy 


issued OFFP Pamphlet Now. 4, which embodies the procedures 
stated in AFR 400-28. [hem eNavaleraGalitaecs “PnAginecrin 
Command issved it38S cwn maintenanc2 manual, MO-327, which 


calis for performanceé-orianted statements of work but dces 
Bet 28piemene the use cf MIL=-STD-105D. NAVFAC is explcring 
“He wse of another statistical method, confidence IJlavel 


estimaticn, which it ¢xamined later in this study. 


2. Military Stancard 105D Concepts and issues 


14 
(D 
(D 


éfo Kee hG Specific Air Force SOC policy, it 
Mem@iiececea-y tO eiabieczate On {he SasSic Statistical concepts 

oes PE 10'S). imese —aSe Dased of acceptance 
Beamiplego and 2-e deScrtined in Simpice tarms by A. Jj Duncan, 
empeeecd SCG ecuchnecr: ~{Ref. 32] 


Reecolipeny Geceayes a Shipment, of goocs. It semples thé 
Sieeedere aha  .e.eher accepts it as conforming eo stan- 
Meda mecebeqec ts 22. if he company reject Ste lot as 
Deemer etew S-andardpeac may be returned to =he eee 
Greet May bre Kept, depending Crile w badly «he goo are 
needed Crwhat arrangements have been nade with the 
Pewee e-teePOsccibiy thers wall be 2 price concessicnr 935 
Mem@eesecmeiccs, §f. 15 20 be smphasized that tne purpose 
eee eGemaeatCe Gatruing iS *0 detsrmine &@ course (0 

peeeueteetec) sO scstgmtece 1O= gualicy. Acceptance sampling 











pec sles a4 precedtre ct 
lots, will give a spec 
given quality. 


applied to a ser: 
SKm@OcL 


Pa cther “words, acceprance sampling yields see oy assu- 

France. It is aise emphasized <tnat acceptance S2Mp aug is 

not an aALTSEMPTt MGont ros ualix Heme ihe Jaetrsr 2s she 

purpese Gt ee chatts; ‘these guide the engineer in 

modiryirg procedures so as to a Oltebe: = emo rOavers. 

Under acceptance sampling, the attributes cf a 

Beoauct are judged. Meee eniites alo eNee, orowerecy Of a WE 
eee. Clessity it 4 


ME c dic cithés within specitie Pana cs, 


u 
y 
So Gecmosrogood. Olicglity chasacteristics of 
e O 

een (OEMZng. [ Ref. 11] Submission of good qualit: 
Meoiulets “22 aigdh fates of acceptancs, while produc 
Gereneyeincur 2a high rate of rejection. 

mee CAT cOntractsS, single sampling plans fcr frac- 
sions defective are used mest frequently. These define a 
Sample size that is to ba taken and a number of defect: 
unats which can not be exceeded in order to preven 
may] Ct. Ch. eket. S2)] AS” an @eample, a sampling -¢ 
cali for a sample of 100 CA Service work items so be tak 
Pom aqumcnenly iot cf performance. Two or less de 


f 
Piece ene. aGceOzance, while threes 59> more lsad ce lot 


fas 
ct 


rejection Such stated constraints lea O e 
eee oh oc. SaakacmeraS@ie CUEVeC; = Which a2ilastr 
b S 


i Eporebsitty of jacceptarnce of a lot 


Mei Geeneagis Acer gh cates of discovered defects, the grcb- 


eevee tec acceptance Will be low. Operating character- 
istic curve profiles cen be adjusted by varying lot and 
Sample sizes or by varying the acceptabls deisct trates 
(accecttarie cquality level, AQL) as shown in Avopendix C. Such 


B@eratsnae Characreristic curves illustrate the protection 
Canes Ce -On Doth ~CCntractics and governm2nt customers. 
Map carton of Mealitery Standard 105D ©equir2es the fclicwing 


sequence of planning activicies. 
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3. Fiannzng Activity Sequsnce Zor MIL-STD-105D 


a. Lot Sizes for MIL-STD-105D QA Plans 


In using random samplin prec2adures and 
BHL-5iD-105D guidelines, Diemo Ze ChemvOrK L260 “LG=S muss 
first be determined. Lots could be the total number 


m 
an a building that are cleansed, the number of vehi 
undergo preventitive maintenance or the number of refuse 
containers that are serviced cn 4 monthly basis. Lo 

accumulated cn octhexr than a monthiy basis, bux shouid be 
Mmeasecici Ve Cr Continucus in nature. Ail lct items should be 


nomogenous, cr have the same characteristics. 
be Sample Size Determinations 


Wineaira (Ge SiZS 2S aCneway =eDL2S 27M EL-SED=1CSD 
are consulted te determine? an appropriats sample $1z2 in 


Der tdetne €natecteristics of the lot. For one ict size, 


there are three different sample sizes corresponding to 
meee Levels of inSpection intensity. Level I is utilized 
when smaller, or reduced, sample Sizes are sufficient; less 
@eecriminaticn is necessary. 

Powe bl 2S the portal Level of  Lnretecttron 
intensity. Sample sizes derived from Level II tables area 
used most frequently for CA contracts. 

Level III's larger inspection samples are used 


(D 
in 
in 
fu 
v4 
— 6 
~ 


MeetieNOEe discrimination of product guality is a 
W 


mw 
mM a 
Re 
ct 
MD 
if 
1) 


+ 
+ 


BesUleing an tightened inspection. These are used 


Bee Majet Ckseeved declines in preduct qualiity. 
Swe ce ols warcron OF AcCepzabl> Qualicy Levels 


The next activity in the sequence of désiaqning a 
Gualicy assurance plan is the determination of an Acceptalbie 
Ciee2tzy Level (AQL). THs variable is described in 
ae to-igon as ftollcws: “{ kef. 35] 
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Mime Act i the maximum percenc defective (or the maximua 
Numkger cf defects per hundred units) Ellas, COD -he 
eee ss of sampling inspection, can be considered satis- 

G@ewoly 22 aw process averags. When a consumer designates 
me@emoncctr. Cc Value of AQL for a caEfain defect or gzreuc 
MeesaetectS, he indicates to +he supplier that his (the 
GWetomer*s) acceptance sampling plan will accept tné 
wecas Ma icrity o WOtS OL Ddeenes eghet ghe supplaeE 
Submits, provided the process averag? level of fpetcent 
defective (or defects per hundred units) in =hes¢ icts 
Or batches is no greater than z=he designated value of 
AQL. Thus, the AQL is a designated value of percenc 
defective {or defects per hundred units) that the 
Custcmer indicates will be accepted most of the time by 
the accertance See an procedure to bs used. The 
Bees FO, Fens provided neréin are sco arranged that the 
Megteactiicy of acceptance at the AQL vaiue depends ufon 
the sazple size, being generally nigh2=t fcr_ large 
Samples than for small ones for a given AQL. The AQL 
alone dces not describe the vorotection tor individual 
Mees Gr Datcacs but more directly relates £0 what m>ga ~ 
Bemscxccected CrCcm a seriss cf lots or batchss, previded 
ene Sceps indicatsd in this publication ars takén.. I? 
1S necessar tO refer tc the operating characteristic 
GUEVe of =he plan to determine what protecticn the 


consumer will have. 


eeomeGesagnation Of an AQL Shail nct i 
Saeeitet bas che Fight =o KnowZngiy supply 
Mrrt Of Product. 


The AQL te be used will be d 
Dy tn2 responsiole authority 


The values of AQL'‘S given in these tables are known as 
pEefesrred AOL's. ie Or acy eae die Ole = Ged -— 
WeeeGecencr than a frefearred AQL, these tables are not 
eoplscacble. 
bamican states tne fclilowing in Telereace tc the 
meee «6f REL. 32 ] 

Momeateeyine | ene MLl=STD— 105D tt is expected that in_a 
Pourerence betweem a Stipplier and a@ military agency =t 
Wiil be made cleaz> to the suppiisr what, for purposes cz 
eccepeance Sampling, the agency considers ZO 292 2cccp= 
Pei), JovVelS fOr product cheracteristics. 


Weewnea secet 12d AOL and inspection isvel inten- 
eee ene Saliple S2aze Can bs dstecmined by using cables 
tven 1 PI L=SstD-105L. These also allow a determinaticn ofr 
Heeeceteanc  feyect BUMDEZTS from these tables. The accendt 
Mer se an 2Mpcscan= th=eshold; as tong aS the numo¢r of 

= 


re 


dzfectives found in a sanp 


a2 





acceptec. If the number of defectives are 
eae aa Or greater than the reject number, aa Loy is 


d. iInsp¢ecticn Schedul2= Design Using Randcm Numker 


Tablés 


The next stp in tne quality assurance plan 
Mestgn is the choice of items that will be includ2d ina 
Sample by using candcm number tabies. Th2 process begins by 
meeregdeeag ©€ach 2eom cf work an a certaia functional cateqcry 
(such as sérvice of heusing area trash containers) its own 
unigue inventory nuater 

A czandcn humber tapdle is then consulted. 
Correspendence between werk elements in the iot and digits 
diszed in the table is astablished (a aumbering sysz4n). A 
Beu~c througa tne table is Selected and is followed by 
mmemoscing bumbers according to this patt2in. "& starcing point 
Mer rrxsa, aed tne table is used until the required number of 
Sample items 1s chosen. 

Mm using sandom number taples, the selecticn of 
Mmespecsacn Sample items occurs such thet each has an equal 
Chance cf being included in a sampls. Detagled, sexrize:= 
[esthLuccicn=S OR randca number table usage are orovided in 
Merk 400-25, OFPE No. 4, and WAVFAC MO-327. Dae {eke = 


mesitanec=s ~inpstectOr 1352S Each 2temn orn a schedule and 
Gemauets anspec=tOns at the appoin=sd tine. Results ate 
Peealmenasd aneWrating™ fo aid in determinirg the accerr- 
Poe ley Gierch-acceptebi lity cf contractor performances. 


@¢. The Dispcsition of Inspection results 


After inspection findings are ccmplete, altéerna- 
Seve sCeturcce= Of actzen may be pursued by the activity 
Gomer act (Spee re Oneal v 15100. ®hen AaAQL's are exceeded, 


eayment deductions er lot rejecticns may be nade. i wor o£ 


b+ 


a 








mma CONcecutive lots are rejected (AQL's are éexceededc) 
eager enecad inspection iS instituted utilizing larger sample 
sizes. Rejected lots may be resubmitted after dsiects 
See served, but this is @one only at the discreticn ct the 
goverrment 

When ten consecutive lots subject te tightened 
inspecticn are rejected, performance is halted. =. 2S 
concéivakie that a termination could ensue cn or befcrs his 
Bount. fice 15. dq ErOVA SlOnMEEOT €xcepeiemalie comeraczer 
performance. When ten consecutive lots subject tc nermal 
anspecticn have been accepted, a switch to reduced inspec- 
tion (Smaller sample sizes) may be made. 


f. Overview 


Tf guality assurance plan designs based on che 
preceding séquence cf activities ar2 conducted croperly, 
Mib=STD-105D will BEOvVide for sarfective and reliable 
weymeitlance Of CA SErvVice contracts. 


The concert of accéeptandis quality levels must be 


! 


@eee-tully Undezrstoed. It 1S not an aspect of specificaticns 
miswecasa ©OcLLacaenen tO Gon -2s¢- 
eillance plans will allow nc amount 


ee= “hat "gevernment suryv 
hy +han the AOL 


defects ckserved in the sample to be greate 


tj 


i 
specified by tne government. [{Ref. 32] 


meee TOsfcmmeenmOnly ussad tanle in the stendard is 
memsingic sanpliimg clan {Table If-A). It is designed so 
miae  caenc diagon@l cath in the table, the product of ths 
AOL and che Sample size is nearly censtant. Piss a iaS 
aeouee-de a d Life cec Set Of AOL'S which aay be utilized, as 
Snown in Appendix 8. 


Ome comenecism Of MIL-STD-105D is that prebabili- 


ti¢s cf acceptance increase as sample sizes incrsas2 for one 
Given level cf AQL. When the standard was being formulated, 
some industrial engineering expe p>2essed for COMsz ant 


5 4 





meebabiigties Of accertance for any given AQL. This nection 
was rejected sinc? it was believed that ubstantially 
greater risks area pesed to suppliers of Ws lets (and 
corresponding sample sizes) when these are submitted for 
inspecticn, than for suppliers of smaller lots and samcles. 


Increased sample sizes actually protect both governmen« and 
cContractcr ianterests as these allow more accurate ass2ss- 
ments of sample and lot qualities. Theretcre, increased 
probakility rates cf accept: With inereased lot and 


nce 
sample sizes is logical. [{ Ref. 32] 


PetseenoLce fiplomenwat-On of #IL=STD=105D in CA 
Gontracts 


In addition te statisticai sampling ~téchnigques, he 
Airc Ferce employs other surveillance methods. Oné of these, 
a management information systen, 22) Ue zed) 3G PLO te Easy 
Me@ontalNenne COh<ractor*’s pertermance. Information supplied 
by this méthod may obviate the need to install tandon 
sampling inspection systems. Such systems may be reports 

Slegited Hy contractcrs, or by government customers who 
receive CA centract services. 

Survelliance checklists are another method cf CA 
surveillance sugqsgestéed in AFR 400-28. Yowever, this systemis 
not réeccmmended if a management inrormations system cr @ 
randem sagpling system can be instaii:2d, Since checklists 
are a icrm cf planned sampling and may be subjective. 


Fermal custcmrer complaint systems provide supplemen- 


ct 
p) 


Sy information describing Conese 202 perfcrmance. 
Gusteomez complamnes, under Ait Force policy, are seldcm used 
See ejecelng sSselvleceS Sf Making payment deductions. When 
Tandom sampling systems are in 2ifect, these cannot function 
as Surstitutes for random okservaticns, Dut may be used 3s 
supplementary documentation. G1 lines given in AFR 400-28 
Peale uwemmacma yi tye tC properly s¢€t up a customer complaint 


System. 


a2 








ACL's are standardized vaiues which are jeinely 
determined by the field activity or its systems ccmmand 
headquarters. AS an example, 4cClelland Aizc Ferce Base 


a a 
Sey2Sicns submit prcspective CA sclicitations with d2sized 
eC 


iz Force Legistics Command (AFLC) neadqua 


choices cf AQL's, cr recommend changes. AOL sc 
MIL-STL-1052 tables must be utilized. AFR 400-28 c¢ 


i 
Bay=on, Chic, AFLC Feadquarters either approve the a 
i 
ene 
Melescleccacrs that ere realistic in helping tc secure @ 
b 


Minimtm quality cf service, since no service is capable of 
being perfectly pertormed. These ara «her communicated to CA 
Bea=erac: bidders ice iS el sen. 2 co led =ne ex formance 
Fequirements Summary (PRS) as shown in Appendix C. It lists 
each element of required verformances, mic Seakdarad som 2cs 

Vieation from this 


performance, the maximum 4@ilowabie ds 
requirement, and tne method of surv 
adjudce performance cuality. [Ref 39] 

The contractcr determines tne 
SMaGzvidual category of work in rel 
conz=ract value, entering these on Per 


Summary sheets and returning chem with th 


t4 
= 
— 
iD 
(Qu 
te 
) 
| 
fv 
~ 
be 
ry 


package. Such percentages are late: 
tions for unacceptadle pert ormance:. 


moeshnowld be nO-e6cd. that’ th 


(D 


USe Of Nil—=sid—~105D- nay 
pummeweCESexVyeqd GdefScts sates Greater than tne specified AQL 


when the ccntractor reaches cr exceeds the reject Aumber. 


For exanple, gGeuprocer en .2000 > 22=m= 15 checked with nernmal 
Eampiacngd snzens&ty (Level If eiawa 1. AOL, Ofc 10° percent. A 
sample of 125 items is required; ne ict is to be rejected 


Mew 22 G2fects are Gnhserved in the san 

later discevered, tke observed defec 

tne ACL (10%) and rejection would o 

effects greater than 13 would cause the 

mememekececdethe AOL cr 10%, this auech 
Ss 


memrs accepted. 





Mhe @ir rorce has adopted a “policy of éssiscing 
@emeractcrs in developing their Own quelicty SCH Lo i 
meeeimams, Accordingly, activities who issue CA soliciteaticzs 


Smemmoce a copy Of thé Qq“ueiilty assurance plan. This éencicsur 
mommarked as follows: [ Ref. 39] 


Mm 


Pomona rOrMa.iOn Purpeses Only. This Quality Assurance 
eueverttiarcs Plan is net part of the Request for 
mmeoecsad! Coa Lhvitaticn For Bids nor will it be made part 


Se any Tesusekting ccntract. 


PeCetecac. AGMen=Stretcr plan Gah also be enclosed 
meee the SClicitatica, G=escripeng Contracs Administ ra<o 
duties in evaluating the performance c= Quality Assurance 

rs (who inspect the CONT SAC 202). Geacrace 


<Ccrs) Way also make tandom inspections of contract 


Ccntractors aré provided appropriate page a6 
Beeo-SID-105D to assist them in establishing their own 
Sua@ibaty ccn~rol systens. PASTEUCSL ONS abe Srovided Phica 
describe tow e2xtrapelated deductions will be made when 
Sp2ciitied ACL's are excetcded. The contractor is never gaven 


PeleduieS CL inuspecticn which nave pean developed from ths 


Fandaqdcm® naumber tabties. 


The Alife FOECe approach appears <+t0 be ons CL Teascn- 
ableness. Conversations with personns Hoes 


tases re ede. nat PertOnmancs <r 


< 
iD 


@ 
mech sta +icai surveillance mnezho 


c 
Gemete OCCULT 2ne4theminitia 


3 
Fost fr rimaiSi226n Detiods st 
Memeseeceo (tae TitS= ons Or two months or osrficrmancs). 
PeL"S ECr mcst comtracts at beth installations have rarely 
keen exceeded. Government perscnnel seemed t be pleased 
with the cesults of random sampling inspection asetheds. 
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Scmé deviaticns from MIL-STD-105D are déetects4 in 
memruu-2ze. NO GEOViSiOn is made for tightened inspecticns, 


4S single sampiing tabies in the regulation address only 
é 


+4 
4 
ee 


nocmal (Level If ; average gquality) and reduced (Lev= 
good quality) inspection. 

Switching frem normal to reduced inspection under 
AFR 400-28 is allowed when fcur consecutive lots have been 
accepted, and che number of defects is less than one nalf of 
the specirtied acceptance number for normal inspection. The 
dGivisicn manager and contract administrator must also agree 
EQncu]. Switch. fir-scip—105D allows this Only aitezr) ten 


consecutive lots have been accepted. PeGe ul bee eG, Ge lak 
anspecticn «ill be implemented if the acceptance number for 
reduced inspection is exceeded. Under MIL-STD-105D, & 


D 
5 
Switch tc tightened inspecticn is necessary when two o 
= 
v 


consecutive lots are rejected. A switch from tightened back 
to nermal inspection is allowed when five consecutiva lots 
are accepted. AE ReeNOU- 26 )dOcS NOt ~2WCOLPOLTat> a sinidas 


Eeovicicr. 


It might be reasoned that thes Alr Forcs déviaticns 
Erte a u 


are méant ct EEC CCS2ci Ve MOCiVaAatLOn tO CORTEACTOrs in 
performing satisiactcrily. These modified procédures are 
Bernmg Weilzzed at most Aiz Ferce activities undergcing A-76 


a oO 
ms Contract evclutions. 


E. NAVFAC USE OF CONFIDENCE LEVEL ESTIMATION METHODS 


The Naval Facilities Engineerisag Command (NAVFAC) has 
been heavily inveclved in the management cfr a growing number 
ela 


Peek cOntcaces fob facility ted services at Navy activ- 


a 


Geies. fc is One of tne fixst syst3ms commands under the 
f Naval Materiai (CNM) to address diverse issues in 


° 
WVebsctcns and begin tne Ecrmulation of policy to deal 
them 


aio 





MavPAC published the 40-327 to offer guidance <c Navy 
@Ct2vities in preparing performance-oriented statements of 
work and guality assurance flans. Thes *guldancs is si@liar 
Memcthat pErevided in Office of Federal Procurement Peclicy 
Famphlet No. 4, Gee LOWwS 81 sbecreom methods cC=her tha 
randcm sampling. NAVFAC has issued 20 standardized state- 
Menzs of work with matching quality assurance plans which 
Seemece cazicred by Navy activities to incorporate specific 
local requirements. POEFwsGns Of 980-327 aezS illustrated in 
Bompendix C. 

HAVFAC has not adopted the use of MIL-STD-105D as the 
basis fer its statistical sampling técaniques, and does nor 

aliow Navy activities to maxes extrapolated deducticns 
sed on its usage. Sampling tschniques may be used at 
e Navy activities which demonstrate the ability to 


Ss 
establish scphisticated quality assurance prograns. New PWC 
n 


mums portat>On CA solicitations being issued in this fiscal 
Year wiil includes nocdified sampling techniques and extrafo- 
jlatec deductions provisions based on their usage 

MAVEAC Pphalesophy is that CA contrac= surveillance 
should be baséad on the statistical estimation of defective 
items in Saoes , moener ahah on “he accept/reject 
mapetresiS testing methodolegy of MIL-STD-105D. Fotitac 260 
Meeanmecenced t0 inject @ highar level of precision (ccrfi- 
Pic -)e Pm cic Siloveitllance process than chat crfered by 


tre > 1D 1O5L. 
Pies CAOlOgY s2=dquiz=S=s a designaticn of cesired 
confidence levels, a relative accuracy of estimation, fpefu- 


lation sizés and a tar2shoid of contormance (equivalent =o 
g 


fu 


meomenelmeCCrcopt). Ccmbizing these elements results in 
determinaticn of sample sizes and corresponding lower ccnfi- 


BewGes 2h ieo te jecercn numbers. For example, a work function 
is performed 2000 times in a month. A confidence level c= 95 
Pemcetems = G€SIlIGCG in estimating the number of pefulatior 


aye, 





Mereects, Wich a relative accuracy equal to 50 percent of the 


+t QQ 


designated AQL of 10 rercéent. (Relative accuracy S 
5 percent). AP cample Size Gf 932 Zesems iS Fequired, and 
Pemgeceren Of “he month's performance can occur if more <ha- 
16 defective items are found. Sixt2en observed defects will 
Becglt in proper ict rejection since the estimated lower 
contidence limit percentage for nenconfiorning items is 
weeways Grecater than the specified WOL of 10 percent. 

NAVFAC has worked with the J£ffice of Naval Research 
(ONR) in developing a set of new confidence leval estimanion 
tables which can be used to design CA contract quality assu- 
rance systems. These. tabbes Gwrdi retlec= three Gaiteren 
levels of inspection intensity (ccrresponding to varzicus 
confidence ievels), known aS tightened (99 vercent confi- 
dence), Boetal (95. pezcent), and reduced (90 percent) 
Sampling. It appears that these new tables are based cn the 
Mmaseccecmetzric statistical distribution. 

iieaenew  Neshodclogy emanated Siace NAVrAC dcubes that 
BEL-~SID-7105D suitably estimates [ractions of nonconfotrning 
@ctivities with reasonable accuracy. Tais was illustrated 
Pee Je When observed detrect rates at the WIL-STD-1C5D 
ect number were compared to the AQL's. NAYFAC ase 
Seen pet y in CchoosSang AOL's other than thcse 
@egeee «nc Standards tables. 
orimary 2mphasis in confidence levei sstinaeti 
Meeemmane che actual cccuzrence of defects in the p 
based on sampie observations whereas Aypoothesi 
Bea2kutes crly determines 1£ populations are 
unsatisfactory. 


th 


NAVFAC has cther reservations concerning the use o 


fu 


Miee-olD-1052. ft does noz believe that full payments shou 
be made when the cbkbserved defect rate exceeds the AQL 


¢ 
4 


rh 


belt Centags, even the reject number has not been excceded, 
and questions the use of the standard where lots may act be 
m D 


ogenous and Siottec=a4 in a2 COHMtTINOUS manner. 
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Other differences in Aiz Force and NAVFAC ccntrécr 
policy were observed during the study. The Air Fores 
encloses copies of quality assurance plans with CA contract 


solicitaticns; NAVFAC contracting activities have no« dene 

so. Aix Force ccatractors are notified of desired AQL's an 

the type cf inspection method used in verifying these. 

NAVFAC will provide this information tec prospe 

fumes cs tuture CA solicitations. Thé Air Force 

Bemagenen:: Control program in Which cognizant Cc 
c 


Reministratcrs cenduct random checks or both inspe 


eA eee 
Samples Sizes For MIL-STD-105D and Confidence Intervals 


j 
| 
| 


| 
| 
| Loz 3223 Sampl2 s22Zes 

i 
| MIL=STD-105D Confidence Inzervals | 

| 
{ 2.0 8 of | 
| 100 ZO Bs | 
| 5C0 50 198 : 
{ 1900 80 len 
| 5000 25 iS 
| 10000 2S 136 | 

{ 
These samples sizes are for normal insp2ction 4t 
{ Soe ccrtidadence, at an AQL of 10%. 
ara ane 
MenmelacGece Lol = crmance. evr AC nas | CoOnsi@es=¢) Sucre 2 
Peegsam EUs has now fcsmerly instituted one. 


Dmcmues-= NO=sseceable di fieztence an inspecti 

phies is visible if sample s Ss é 

micseasing lect sizes, as she 

ieas Ull—-StD-105D acre less then tho e O 

€stimaticn fcr smaller populations. Two inferences can be 
g 


—S 


ieawh s@CM this fact. PEM COcE Sov. 4 
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Meer cance in conducting CA contract survsilléence, teen 


MIL-STD-105D sampling tables should be utilizad because 


Smaller numbers cf required inspections result in less coscs 
mo che activity. However, if BelstOn 25 adciyudging 
contract performance is essential, then mpling under 
confidence level estimation should be utiiized. The larger 
sample sizé¢ provides greater reliability in assessing the 
number cf defects that might be submitted. ipnere cs tS ier 
mespecticn are nct great, this method is preferable. 


In one study at the Charleston Naval Shipyard, fran 
Sampling precedures were used successfully for daily, rtepet- 
ieeve Services. All inspection personnei found «hese to 
Supericr tc frormer heavy reliance on planned sampling 
methods. Additionally, use cf such techniques resulted in 
peepieacane lowering cf observed defect rates in conzractor 
performances 

The authors were told that an informal DOD working vaneal 
of Air Forces, Navy, and OFPP personnel will begin an ev 
tion cf aifferent available statiscical methods when the new 
mmercampleng cables are complete. Until then, che only crts- 


Mea!) Statistical methcd availabls =or use is MIL-STD-105D. 


Fo. USING STATISTICAL SAMPLING RESULTS FOR EXTRAPOLATED 
PAYMENT DEDOCTIONS 


As well as promcting more optimum use of costly insnec- 
eon f£==curces, SOC etecudegisS ~ DEGVade another  imocr tant 
cee: GeSultS Can De Utzlized in extrapolating, or 
epolyaong the percentage of defectives found in samplss =o 

he larg 


Plec. pierulad =_Ons. 
Dye eererorece first Mandated this procedure in @aFR 
bee 2eameehehe Ssallpie Errors occur at @ rete which is greater 


than the desired AQL and its corresponding reject numb 


-= 
— 


= 
fv 


2 
Be-ecentag= +cund Unacceptable results. It is subtractec from 
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100 percent to determine the loc percentage 
payment, which is applied to the entire lot 

not taken when errors in a sample are les 
number. 


NAVFAC policy has not allowed extrapolated deducticns 


Bmece i. nas NO Crficial statistical method on which <0 base 
these deductions; however, pregress in déveloving such 
metheds has led NAV?PAC to envision alternative applications 
Gi extrepclated deductions to be utilized when a uniform DOD 
pee tSstical Sampiing colicy is aun Append:x £— iilus- 
trates varicus Air Force and NAVFAC meéetheds cr makin 
payment dedvrctiors fcr nonconfcrming services. The NAVFAC 


metrods have not yet been premulgatead as new CA contract 
BOlLCYy. imecovowlhd ce posed thas NAVPAC wili deduct tor all 
tteme chserved to be ia nenconformance. AHoePSON Loa cGe.Ges 
satisfactorily reperfcrm these within allowable time ‘frames, 
nOow2ver, Speci -sawill, be given for =fSenertormed items. 
Liquidated damages are ass¢ssed as a percentage of all 
Pemechicr@ing items ce work. 

Ppemmlgeticn of a4 Uniform DOD statistical sa 


M 
Meee y Wail Dlay an important rola in alioving the use of 


D ts 


meecdrpolecsG deducticns based uoon this *t¢chnique. or 
a 


in 


ampling prccedures are perceived to be unreliable, Jliti 
tion proceedings may eventually prohibit the use cf ct 


deduction methodology. 


G. AWN OVERVIEW OF SCC TECHNIQUES FOR A-76 CONTRACTS 


Mememcwide be eptarent that ths use ci Tandem sampling 
procedures in «he new A-~76 Commercial Activities p 


u n 

DOD and other federal agencies offers signifi 
savings in emeract sadMinistraticn. ihspec =. On 
reduced, ard ae of $QC techniques nay icwer the 


Se iomeitomeconssacc OLS NCU in peritorming such Services. 


6 3 





Meese techniques cfrer QalLEnSSS “aud Gbyectavyity €haé 


eiten were not present under former planned sampling and 


custcmer complaint prcgrams. If SQC tachnigques are adminis 


Meeeag pscperly, they will render a much more accurat 


() 


W 


pp 
(7) 


= 


presentaticn of true contractor performance =ner 
possiktle under fcrmer inspection methods. 

Such techniques will be invaluabie if large, mnultifunc- 
ti0n contracts at federal activities become a reality (such 
memeone =Ctal base Operating services contract at the Naval 
SaeM@arine Base Bangor, WaShington). These will ovotini 


Zi 
use cf girnimal government contract administration resources. 
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Ve EFLANNENG AND EFUDGETING FOR CONTRACT INSPECTION 
ORGANIZATIONS 


Aw. INTEHOCDUCTION 


With the implementation of OMB Circular A-76, DOL activ- 
ities that study CA functions must prepare for the possi- 

Pyeenc ad CONtract award by identifying and plann 
certain Support requirements that would be neéecéessizctated. 

Sa plans include a@ quantification orf inspection resource 
Bomeonds the activity'’s budget can accurately reflecz fund 
Beguieements for either the creation of a service contra 
Segenizacicr or for the augmentation of an existing cne. 

The Commander, NaValb Facslicies engineering Connaad 
(NAVFAC) and the Commander, Naval Supply Systems Command 
(NAVSUP) Mave veaGditvonally held  coOmcsace auzhnerity for 
constructicn and procurament of supplies. Botn have dele- 
gated contract authority to régional procurement commands 
such as NAVFAC's Engineering Peci de aDiviestons ~(RED™s)., 
MPePSUE'TS Navy Regional Contracting Centers (NRCC‘'s), aaa 
Naval Sucply Centers (NSC's). Thes@ commands have further 
Pweg d Ceimeace sau hObitweto, fi2cid activities. Alcng with 
mies authcrity , field commands have been assigned the 
Besoensibility for quality assurance and surveillance of 


u 
Benstruction and delivered sup 


plies . In the Naval 
Meemlaestecs SHGIRsering Command, th2s2 are known as Resident 
Paeeccee In pena ge cf Construction (ROICC). hese fleid 
activities have been staffed with necessary quality assu- 
Tance and contract spéciailist peescnrel to administer 
Sens elvuGclcnh amd supply contracts. Additionally, the 


Peemoustomlity Lon the preparation of contract solicitation 


Eacwages (66> CxXample, the design of construction projects) 
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has usually been assigned to the regional contracting cfiice 
memecoetns =6qQuUibing activity. 

Mies Udteerh Of Con<ract authcrizty and surveillance 
responsibility does not apply, however, Geng (Gri 
contracts. Nelther NAVFAC nor NAVSUP have been provi 
sufficient staffing resources to provid 


(p 
O 
a 

t 
W 
2 


1 
Memice 2nd edministration o£ contracts resulting from A- 
D 


cost studies, although both systems coamands rc 


(a 


meenecraty £cr CA contracts. Na VV aGesv. -16c0 05h 
Services under these contracts prepare the s 
work, guality assurance plans, Seer Oe Vad are 
Surveillance organization to ensure Peope> Centrect 
Be=fOrmance. 

A Chief cf Naval Crerations message that was promulgatad 
meeeze November 1982 delineated diftfersn= roles t¢r beth 
COn*racting agent ccmmands and customer activities in their 
mapeementation of the CA progran. Lieve Otloweiag “resvGrsi= 
Peectdes acs identified: { Ref. 37] 


Gumecen*bacting Of fice 

1) Peecessing ©f. Concsac documents which requires 
PketGise Ceecontlract authority, 

opi) NedOciatsen Of all Contract Changés, 

ie yebce Ct LomeoL remedial contrac cor action, 

iv) The epEOocSsssiugmol Contractor payment requests, 

v) Boelecaeecn OL authority to =h¢e customer activity 
SOL any cay-*o-day surveillance ox the contrac- 
Zoe Ss HOLK DSELoOrImance, 

Vics) Maintenance of Pe eOr a iV Sold Lous tas 
rrocé¢ss, 

meme) PereCVYision Ce all techzrical advi 

Veva 2 )eePEOViSLOnN Of aszsistancer in th 
tSaining programs for all versoennel invclived in 

a 


contract prevaration and administr 
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b) Custcmer Activity 


aL) Provision of qualified personnel to i weSSK 


| 
G) 
tc 
a 
OQ 
cf 


@eliweced under the contract, 


j- 
j #- 
= 


Preparaticn and implementation of quality assu- 
rance plans 
iii) Submissicn of quality assurance summary terorts 


Eo. ke Gentraceamd agent, 


iv) PieeMiaee Can (GCCR-SaCecoL TSQUEG 2S Lon peynent, 
v) Recommencation of payments or d¢eductions tc the 


Gone facting agen, 
vi) Submissicn cf cost estimates to the contracting 
Orel eGermtcrn DEOresed Medi ticaticns, 
Vii) Provision of assistance as required <=9 the 
SeesaccirG sagSie Gusing Node fication  neqcetia= 
TLOEes. ard 
viii) Perfcrmance of o is 
duties when these are Gelegated by Tie 
Cie aaccing ‘OL LCe s 
Meee bas aletrteaq all Navy field activities a 
Pee Nenes Of this division in service conzract responsibil 
Mees 1) Hcth an instruction (NAVFACINST 4330.45) an 
mMainzenance manual (MC-~327). The Alr Force has ad 
Similar aprroach which is specified in twe sevara 
Force Regulations (AFR 400-28 and AFR 790-9). They require 
mde tie functional divisions of each Air Force bass prepare 
toth the statement cf work and corresponding quality assu- 
memce weoban. ihe division must provide all necessary quelity 
PaieanmeG-mmcesSOnnel and, prior +o she conzract solicitaticna, 
Ge=2tifty 2n writing tc the base Commanding Officer chat these 
onnel haves been designated, Sealed ~meeaG  d 
ly <tc perform surveillance functions. [Ref. 38] 
Romeemo Air Fotce installations visited du 
tes¢arch phase of this the 
Cc 


S Ww 
lidated steering committee is organized up 
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memouncement Of a CA cost study. It includes representa- 
tives frem all departments that might be affected by the 
Bemcraccing out of a function and tailors standardized Ai 

Force statements of work that have been prepared for ¢ach CA 
mancticn, fdettimyeng and “daeluding a4) special local 
command requirements. The steering group also frovides 


assistance in the development of the guality assurance flan. 


Bo THE ESTIMATION OF REQUIRED INSPECTION RESOURCES 


At the cutsec cf a cost study announcemen=, an activity 
must begin +o define and plan its quality assuranc? réequir2- 


menis. (Ref. 39] The planning and budgeting responsibtiiict 
for crtaining inspecticn resources belongs to the activity. 
Each Navy activity shculd immediately bagin the planning fer 
Meo crac. ACM=2=nistretion staff, establish ongoing interface 
Meen <he approrriate major claimant <0 document these 
B=GuUitSsments, ¢eStimate future budget anounts, and obtain 


Peerage CCintsS <O facilitate the creation of £his vital 
a 


SeJanizaticn. 

YVaricus procedures have been used for determining che 
required Guadtticyv sect Seevac- (COno rect administration 
resources. Pam omeuipplement Of che 1979 revision of A-76, 
a WwW 


u iF 
as stated that ccsts of administration resources were to 
Scere sO uhOMt: Des Cen= Of che projected award ces 
Wee COnt rac t< DaasecadGtos 25  Usstiul oniy as an 
t@€ 2stimate of inspection requirements. Actua 
MeCcOs “omg nt ss cange Erom can percent fcr small 
Semavice contracts <0 only two percent for large BOS 
Semtede=<s., Use Of thas simple €stimnating factor does not 
Become £Cc= diftiering degrees of complexity among contracts; 
however, i+ may be satisfactory as an initial estimating 


“Oe REL. 5 | 





Simple estimating model for determining inspec- 
s as been recommended by the ‘the Southern 
Eased Division (SOUTHDIV) On he Navai 
Facilities rtngineering Command. I: recommends that activi- 
Peels Can Plan to use 875 hours of inspection for ev 
mg00,C00 dcliars of rarfuse collection service ccntracts, 
while all cther service contract types raguire et ie 


Mmo2<> Occurs of inspection for every $300,000 of contract 


TABLE II 


New A-76 Centract Administration Pactors 

















— ... ere 
| in-Hcuse Staft Ecsitiors COmrmact (ACM ELStzatzcn. 4 {| 
| Being Studied Szatfi Requirements 
| Below 10 USS 2exXistizg staftr jj 
| 1O™m=e5 20 1 
21 - 42 2 
| a) Oe 3 
i 66 - $1 4 
| 92> ie 5 { 
Oa 1.9, 6 | 
| TS i= Vea if | 
{ ioio eee 2 8 { 
| eee CS 9 | 
| ZO Ome esi iZ 10 | 
313 = 367 11 | 
368 - 429 ez 
430 - 5C0 es 
BO} = 582 14 | 
584 - 682 Es) 
6535 = 7600 16: 5 
Above 800 Com@e Jn-HCUs] 99-225 (S500) 
| . ea a = ss 
eests . This model, like ths previous one, 1S primarily 
userul for making rudimentary projections OL Service 


cOrtzact inspection requirements. 

The 1983 proposed révision of Circular A-76 frovides 
guidance shewn in Table II that relates the number of 
Pees otcuereduir=-d ECcr contrac= adminsstraticn to the number 


of personnel positicns béing studied for ccnversicn. T= 
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Pim iew =eomcscimdalLe the Fequired conzract office stafiizg. 
a 


This particular methcdology, as the two previous onss, nay 
Hece paGGOunt BOS differing complexities which e¢xist in 
services contracts 
es ee a ee a a ee 
Sc = 69.74 + (0.183) (A) + (7.29) (8B) ! 
| Where A = Total Service Contract Vaiue (3,000) | 
and B = TIctal # of Sa@rvice Contracts | 
| QAB*s required = C divided by 144 | 
| 
Span rm arenas, 
Figure 5.1 Inspector Hours Regression. 
Ancther methcd for estimating the number of required 
miIeeeccOnS 1S) S6und in the Student Guid for Maintenance 


a 

Sevice Contracts puklished by th2 Civil Engireer Corps 
@fficers Schdol. This mathematical method, depicte aon 
Figure 5.1, iS an estimating model that reiates the number 
cf required :nspectcrs to the total monetary value ané 
Mumber of the activity's service contracts. [Ref. 40] 


The Atlantic Engineering Field Division (LANTDIV}) cf th 


(D 


Naval Facilties Engineering Command has proposed an alterna- 


f 


tive which may be the most viable one tecommended thus far 


Mideerc OLOVedsd as Appendix F. LANTDIV Instruction 11014.4D 
ec 


recommends the use of a standardized worksheet to accurately 
estimates required inspecticn resources. It cseguires the 
Specltiie guanticies cf various work elsmentes to be perfermed 
iMedere2be Centrace ON a mnOnthly basis, estimated number of 
inspecticn hours to chkserve «hess, a20q che tOsal number of 
feoviee@eceonsPecpeinagdsyidual categories as well as the total 


[wien Ge  PELEOLMance £Cr Sach month. [pis csc.lla<aOn Wal. 
allcw a@ determination c& the required number of inspéctcrs, 


based upon a total estimate of inspecticn nanhours. 
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UtemeiitactOneOL his werksheet should res 


Hekc. 29) NCra 
accurate calculations of inspection personnel resources thac 
are required if a new contract is awarded. This appreach may 
emeaowtecagd) =O che Creation of sufficien= data that can be 
used in the development of engineered performance standards 
Eemminsceectacn Of COMMErClLal activities sézvice contrects. 

Tf any of the abcve estimating models prove =o 


SesoladcCeory, them che safest means of 28timating p 
requiremerts may be to predict that one new inspect 
Ec 


be e¢specially true if the inspection forces are 


in 


9 
necessary for each new function to oe awarded. This ma 
1 
£ 


undér a functional department, as is often the case 
Force service contracts. 

Inaccurate? estirates of required scafIing may create 
PeeeeeusmeZOtlems LOL GCclVYiti2s that continues <0 ccnt 


cut under A-76 guidelines. Navy activities must caref 


Qs 

cG oH 
my 
cho 

M< 


esign all of their quality assurance plans and calcul 
Peeesequired “Rumber cf inspectors and staff personnel t 
eeccucw- cicm. Alenough current policy dictates that defini- 
*ized quality assurance plans and organized quality assu- 


rance Steers we be ESt ae 120 before Gen race 


Qs 


~GieeGewawe ONS,nesca sen fOr this study indicated that femal 
quality assurance plan designs often lag contract award. 
MeceVities that were visited frequentiy used existing 
personnei tc provide surveillance on the new CA contracts. 
Acticn ovursued in accordance with these considsraticns 
MeieeemesSehtaal tO Ssafequard tse activity’s interest in a 


o~ 


Teswenoe =scmad Ort  ~CONtracting out [Or the Ne 
Sec cowemeit 2oiS Svcluction 1S not planned p 
fective cz erratic gcvernment surveillance ma 
a cCem@ewonucaa Os a Significant tactcr ia the 
litigaticrs cr dispvtes which arise when the prop 


governrert inspectior is a major issu2. 
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Cw. QUALITY ASSURANCE TRAINING 


Ur to this point, it has been emphasized that the dex 


(Pp 


tt 


Mia soneanawestarting Of a SUfficient aumbéer of contract 
HmeDeGe2O) and “adwinistration personnel is vital to a 


successful implementation of A-76 objectives. An equally 
important consideration is appropriate training cf these 
perscnnel who will ke involved in contract surveillance. 
Fven if no additional inspection personnel can be obtained 
Pomowich bmidcetazry pmecedures, the activity wall stiil be 
meGQbeceagstG 22O0ViLG9e a cadre ct insgectcrs from 4 
perscnnel assets whe will need to 090 
surveillance techniques. [( Ref. 41] 

WasegecnisS teséarch, it was discevered that twe Airc 
FOrce activities designate and train Quality Assurance 
Evaluatcr (QAE) candidates during the statement cf work 
BReDaraticn process in accordance with air Ferce Regulation 
99=9. Current Namy celicy nas lef+ the choice of inspectors 
Secret nanag FEeSPOaSibilicty to the discretion of each activ- 
fave Ss ComMenaing Officer. Naevy guidance in this area (that 
oi NAVFAC and NAVSUP) has been only advisory in nature. The 
cniy writtén mandatory reguirements Levied upon Navy activi- 
ties is that they prepare and submit a quality assurance 
Meroe GOReeaGccingd agents <OD approval prior to sclicita- 


quality essu- 


fu 


PEeiemeGney TUS- also cercify in writing wet 
MaiGcmwOst2OLrco mel ll be established of augmented priczr to 
the ccntrect award. These requirements have been address 
in tecsnt NAVYFAC Engineering Field Division guidance that 
Pewee esr ca SeLVvVice Contracting. 

Newly designated Quality Assurance Evaluators cann 
expected to immediately and professio 
Smmpveitiane@e resconsitilities until va have been gual 


byet@eens cia formal training pro nies tay lrequaze 
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attendance at a special school desig 
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Wigeecyed=sstlance ©Ssponsibilities. It may also invcelve an 
EagoOanrG geo ces ror M@alized training orocess that is caztrisd 
out by skilled quality assurance specialists who reside at 
Sreaauveci# “he activity. 

Service contract training courses havée been design 
are béeinc taught to the grcwing number of federal fe 
who perfcrm surveillance functicns. Bor we ie Ar oP 
training is conducted by a quality assurance program cocrdi- 
nator. The QA Program Coordinator serves in a general advi- 
Sory capacity and wmenitors the performance or functional 
department CAB's after they complete initial training and 
begin perfcrm@ing surveillance duties. QAE candidates receive 
Paaomerardong in Gontract law and administration, quelicty 
assurance duti2es and responsibilities, and an overview of 


the quality assurance plan . The training 


a 


esponsibility for 


ct 
2 
@ 


Quality Assurance Evaluators is placed at activity level 
und2xr Air Force procedures. Quality Assurance Evaluators ars 
identified and trained before the contract solicitation 
process tegins . 

Fea the Navy, cCuality Assurance Evaluator manning is 
Ranated By the activity and the training is carried out by 
jemer Reatest Engineering Field Division. The Facilities 
Division (ccde 10) of each NAVFAC £E 


a 
EO Yeaval activiatszes in its geographic area. Training of Navy 


ry 
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ct 
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Geet sels @ coOntzacting agent responsibility. EFD's also 

Peoviidec kecennacal SUEECrt to Each activity during the state-~ 

men= cf werk and quality assurance deveiopment processes. 
The Naval Fecilities Engineering Command has develcrped 


ZtS CWH training wanval which describes the specific roles 


assigned to quality assurance evaluatcrs. This manual is 
known as MC-326.2, “Quality Assurance Evaluators Training 


Manual" and is utilized py each Engineering Field Division 


im its quality assurance training presentations. Quality 
Ssurance evaluators perform «he following duties which are 
[oecCeewmne=32o,2>5 [Ref. 42) 


gee 





a) R¢view plans and SSavyeCe Seoverace —Speciftca=—_cns 
Demesercomeract SOl-Citation; 

b) ASSist in preé-award surveys 

c) Attend the pre-bid and post-award conferences; 


d) Coordinate transfsx of government furnished space, 
Utilities, 2quipment and material to the contractcr;: 

¢) Prepare quality assurance plans; 

f) FErepare surveiilance schedules, perform surveillance, 
and submit reperts of findings 

g) Review all contractor scheduias and advise Service 
Gantract “Manager ot acceptabilit«y; 

h) assist in the préparation, Olgedirectly prerare, 
goverrment estimates for change orders; 

2) S82¢ecmmend deductions for unsatisfactory work to the 
Service Contract Manager; 

Dieter OSe=-n= “CCN <;ractor’S Safety ptacti ces and Espcrt 
results; 

K) Conduct labor standards interviews as necessary; and 
Meconaue: SUuazveitilancs on the contractor's acccmpiish- 

ment cf£ required corrective changes. 

Pees eid Desa MEnasized that. his siise 1S sot. =sxhaus- 
mao ems pycs Cf an 2npliciz solicy that quality assurance 
evaluatcrse should be dedicated solely to surveillance func- 
Pees,  @hey ‘May wOrk for func=ionai managers and pericrn 
Several cther duties in addition to their surveillance 


m=“ SONSlTEPLities. 


Meee ceomewdecing QAE responsibiiiczies, Hes =Vviden= 
that aweil organized training cours? addressing several 
eLrements er Contract adMinistr=ation is essential. 


iticnally, the ccurse must be presented in a clear and 
concisé manner using terminology that is easily understan- 
dable by the layman. TeciienGeesOasse contract law, 
Soeeueieteaote Glee ciples, COSt 2nd pric] analysis, organi- 


ZAeeOnmecm=ChGre,.) and contract Specifications shculd bs 
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fs 


covered . An equally important topic which must be include 


in any quality assurance courses for service contracts 


v7) 


a 
Statistical quality assurance. As discussed éarliés, th 


aD 


use cf statistical Sampling ‘techniques has keen advocated 
ror the inspection of commercial activities S2rvice 
contracts. It ofrers significant advantages over the use of 
en¢ hundred percent insvecticn and reliance upon customer 
complaints. 

SeeecurnecmenaIn2=Ngd in the use of Military Standard 105D 
and randcm number takies is requirad. Inspectors should know 
hew <c f£crm homcgenouvs lot and sample sizes, establish real- 
istic acceptabie quality lev2ls (AQL's), create mnonthly 
randcm sampling schedules, and conduct inspections. Finally, 
Mieoec Ors Willi require instruction in properly disposing of 
unacceptable variances from required perfcrmance. 
Seetistical quality assurance may cic a ly appear 
conrtusing, tut a well designed training program will reveal 
Reece oe a Much Simplified and useful approach. Randon 
Sampling will be more effective than 100 percent inspection 
cr planned sampling techniques and raquires fewer inspec- 
toErs . Aowey Sede remeae fOr sfileccive service cont=act 
courses is the presentation of random sampling inspecticn in 
@ simplified and urderstandable manner. Different DOD 
training ccurses have been designed with all of these 
requirements in mind; the implementation of a uniform DOD 
Sge=espon@encs course couid further facilitate suceessful 
OME Uiaining. 


One essential requirenert for adequate training 


fv 
+3 


Peeodtal (tems ttverent funding to covet ecocsts of providing 
necessary training tcnew quality assurance evaluators at 
feud activitLes. effective quality assurance programs to 
administer CA service contracts will be created and 
Sustained cnly when activities and claimants identify, plan 


and budget for ccmprehensive training programs. 
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Study observations revealed that several Navy activities 


have attempted to send Quality Assurance Evaiuyators 


ct 
QO 


appropriate training courses but insufficient funds he 


(D 


ct 


Vv 
prevented attainment of this goal. It waS observed tha‘ 
n 


on-site training programs were generally non-¢xistent, a 


jer 


should be initiated to enhance the skills and capabilities 


of Navy Cuality Assurance Evaluators. 


De. INTEGRATING RESOURCES TO BUILD A QA PROGRAM 


Thus far, three EFasic building blocks for an effaccive 
quality assurance crganizaticon have b2en discussed. These 
are the cbhtaining cf sufficient personnel, training of 
GAB‘s, ard funding of inspection resources. If any cf these 
elements aré inadequate, the quality assurance prcegram to 
Support A-76 contract conversions will be jeopardized. 

- 


Perhaps the underlying intangible element that ties all of 


1 


these tcgether is cemmitment. The activity Ccmmanding 
Officer, the base personnel, tne najor claimants, and the 
Pyeeouswconc=acting agercies wili need to form a ccalition 


pomaccuretsily plan ard provide for 
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these building blocks. In the case of the Navy, coopérative 
SeeMbEcS are being pussued to en ao Out successful 
Semenwac-2ng cut 

The authors found WEedeUVatiaewons Pain -ceese SeLfozrs. 
Formal iesign and implementation of guality assur 
has laggec the awards of nanny CA contracts and ma 
inspecticn systems at Navy activities are no 
need Randcem Sampling inspection is us2é at very few Navy 


=eccmo nat oaminister Service cCcontfacts. Scme reascns 


fu 

Q 

t{ ¢t 
J. 

< 

ae 


Ss infrequent usage are that it seems to be tco compli- 
cated and that extrapolated deductions are not yet allcwed 
Cemmtav cont =actS Utilizing random sampling. These Navy 


ActiViItsse Visited Gel oved weohacw de menor measc2on Ccf® a field 


76 





Semesac-I0g SyYStem L¢r Service contracts that approximates 
PucomonrgeneZzecenon £065 CONSZSEUCTICn contracts will make che 
administration of such contracts more viable and é Cc 

The size of the inspection organization and the scrhist- 
icaticn cf the predesigned quaiity assurance plan determine 
the adequacy of the cverall contract administration systen 
Pome CAGORtraCts. Acequate staffing and a scund inspec 
perawiltsle=aq tc successful surveillance, winning the co 
Pweeeeca ehd SUPCOLtE of both the contractor and the 46 
communities that receive services. Pols Sola wor Cpetm2 zs 
the use of inspecticn resources. Effective scheduling of 
Psoeeceors eid audits of contract administration can be 
achieved. An alternative, less optimal a 
GUality assurance flanning to be limit 
resources that are available to che activity. It is likely 
meee tnae CeEtlon Willi be the one most often cheser if act: 
Pee aecenGs DerlOrm@ the necessary pre-contract planning and 


Peeoecingd Ler enh=s £OtmUlaticn of CA contract organizaticns. 
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VI. ORGANIZATIONS FOR A~76 SERVICE CONTRACT MANAGEMENT 


A. MANAGEMENT OF SERVICE CONTRACTS AT NAVFAC HEADQUARTERS 


Preceding discussions have set the tone fer the predomi- 
nant issue of this chapter, the makeup and locaticn of 
PPemMeG-meecontcac.  iInspeczieon and administration crganiza- 
Ome e oes G252St acCrropriate to examine the organizational 

tTructure for management of service contracts at NAVFAC 
headquarters. 

NAVFAC has located management of service contract pclic 


for facility support contracts with the Assistant Ccmmander 


4 


s aiso 
the 
Engineéering Field Division level, where 4a Sitiiac 


Pee Maintenance and TIransportation (code 10). This 


fy 
at 


true for the management of service contracts 


Maintenance Division (code 10) Oversees the evoluticrn of 
Navy CA service contracting. hummer. Ty cOb) dike COhtracts 
MecmmeLG cl eAGtTIiViticS Or construction} moaVec ec - iM. ne 
Acquisition Department (code OQQA) at been headquartere and 
Ee Levels. Meee cal Engineering Frield Divisicn pertoras 
the fcllicwing management and contract duties fcr A-76 
proctremert: [Ref. 43] 


a) 


= - 


Peers buces Cenusacting “dareceuves eae provides 
macomeace a i1e¢ld Contract ele ices ; 


D 
3 
Mieae= heme yity contract ~offices during contract 
d 
Ec 


SEEN cSs 5 
Cpe Gsceesmene Oferatl-Onyof aczivizy contract offices tc 
ansure mewn Lntegs. ty et CAD Ser vice Gor Faace 
rrocesses; 
G@)}) teviewsS activity requests Ecr sole Source precurement 
and refers these to NAVFAC whan appropriate; 
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Symes oS ene central point of contact in the regicn for 
all CA service centract matters; 

De @ammealns as =s<Cchnical libgasy tc support ccurliazncee 
With the A-76 frogran, and provides technical infcrta 

eson £O CUStTONEC activities: 

assists activities in statement of work and quality 

assurance plan development; 

Mycercuccs @AE training programs: and 


- 


i) reviews each activity's statements of work and corres- 


t- 
}-* 


pending quality assurance plans b¢rore contrect so 


(A 


enjita= 2cn. 

Ey ccmparison, cverall management of the Aiz Force CA 
SemvreemCCNerdct DIECGIaM LS conductecé at the headgquarte 
level by the Air Ferce Service Contract Advisory Grcup 
(AFSCAG). Intermediate systems commands such as the Military 
Poeenec CCmManammmnc) Of Al= Forces Logistics Command (AFLC) 
may publish amplifying guidance ‘+o supplement “we prinary 
Pewemererzece  sequlatory policies fOr service contracts, AFR 
400-28 and AFR 70-9, as Shown in Apovendix Hd. (Ref. 44}. 


Toais ocomprensnsive management és alsc visible at the 
activity level when functional divisions develop statements 
Ge Wie = K SNeweente LOL the activity's requirements 


Intern 


eer 


¢diate systems commands granz Pana le oS 
Mey Stas SMents Cay WOrK betcts issuance of contract 
Sel. Cicatkonus. 

ial oem 9S 2 2 & 


c VFAC's Engineering Field 
LDavisicns tkegan to consider the 


onsolidatzics cz pel: 


Recau sup DOL ewtcr CA 


oO 


Rome eC Cie swaccs SLancn. 
Pats head ¢f£ this bcranc may be delegated contractual 
authority that is separate an dost Cc ae eaUlen ers, 
held by tke departmert head for Za fEUGe LOL wena 


m 
C c 

Pave wecakle @emeruitbener delegate CA contract authcrity <o 
ne = 


}-— 
fu 


B2esOcGenesewcOgnizant andividuals at «1 Llével (such 


Ls 





Pope cee Poeaetivaty’s PubliceWorks Officer). This c 
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Moore Uncon tseuay, Oriering Striking possibilit: 
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iQ) 
in 
th 


O 


the future administraticn of NAVFAC's service co 
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Hernmaps NAVFAC w2ll consider she creation of a 
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ry 
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Bemev2ce CCh=tect Grganizaticn that is an exacce liken 


(D 
in 


O 
mt 


S 


O 


the one for construction since the staff and expertise ¢ 
Manage suck a rogtam is largely in existence at NAVFAC 
headguarters, Engineering Field Divisions, and the large 
Pap ae ACEKS Cen tess. Such a New organization would recuire 
the establishment of field activity service contract offices 
Mompecmoce UWitormirty in administration of A-76 ccntracts 
and establish management control of the service contract 
process, and in turn promote NAVFAC's credibility as a 
Service ccntract management agent. 


Be THE ACTIVITY ORGANIZATION STRUCTURE 





fiemeeicCmcULEEnt A-/6. Setting, mest commercial 
funecticns are related to facilities or supply management 


Aecordingly, th=-= officials who are responsible fer these 


“o 


eeeagteat GOSt naval activities are either the Public Works 
eeeecer ce the Supply Officer. SUG MeeuNcclons are vital to 
mleGcecetul “accomplishment oz the activity mission, and 


mel yeemte PEsvVace them in a timely, cost sfiective manner 
could sericusly jeopardize the operating posture o the 
Berivelecy. MOSe cunctions are currently per 

Gv tansmeer Dilacary personnel, Due TeGmowing cren 


Seiecsactergmen- under the A-76 policy Signal 4 change 
2 emeca Public Works Department Functions 


A sample of the different roles performed at 


$v 


SpouGaletlevyeraoanc Wcoks Depariment is as follows: 
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a) 


D) 


Cc) 


qd) 


¢) 


Eaganeerang Division 


=) Prepares and reviews data ror military cconstriuc- 
tion and special projects vorograms; 
a2) Provides design s¢rvices and prepares plans and 


specifications; 

Pewee iecet aces with architect-sngineer con*raectors;: 
and 

iv) Provides general technical assistance +t0 ctner 
command crganizations. 

Maes — Ge ee CCNTrOl Division 

1) Receives and maintains control of work requests, 
facility inspection requests, and job crdezs; 

aa) Pawages a CONELNUCUS inspection pregran tor ail 
command facilities; 

iii) Prepares work plans; estimates manpower and 
materials requirements for job orders; and 

iv) Manages the administratioa Oz maintenance 
coeyece Contracts. 

Hevica ng bavi Si cr 

a5) Manages family housing operations; 

ala) Arranges inspections and maintenance cf family 
housing; and 

feo eenssists in budget preparaticens. 


Pain cena ne] 1 ViSlL Cn 


1) Maintains all facilities; and 
aa) Accomplishes maintanance under emetgency service 
ZedUieswsecr: Specific job orders. 


Cees D2Vi sion 
i) Cperates and maintains all command utilities 


equipment and structures. 


teas ren<~a On Livisicn 
1) Cperates and maintains a noctor pool; 
a) Examines and Licenses all governmen= equicment 


- 


CpecGZacors;, 4nd 


a 





211) Identifies and detarmines transportaticn zquip- 
ment reqtirements. 
This preakout of Public Works Department tasks idéa- 
SELES acreas subject to CA cost studies: maintenance, util- 
ZeLes , and SuansrhObcaticn .operacions. At many ‘avy 
activities, such cost studies are in progzr¢ss. 


Bae) Shevecus GAY EAC GA Service Contract Administration 


BeOeess 25 the choic= cf a suitable organizational structure 
Memeceevl.y'S Contract admsnistration resccnsi- 


to manage t 
Peta ties. NAVP 
n 


Demuae lets che Choice ot thess i 7e, lccatzon, 
él skill capabilities for this new organiza- 
en of the, activary Commending Orticer. 
sked *0 previde surveillance sSincé contracting 
€é not been ‘aioli SUfEsciecnt Siunds cl perscennel 
Beceweces tO Manage CA contracts. Berore 2£hs Cemmancing 

ae should be inicrned 


wes design. 


Saracen bualds such an nage 


See nesce factors which should influ 


(D 
tJ 
O 
iD 


Rieeemona! wee DLiaNning, Controa and post pericr= 
n 


Mance inspection or Public Works aiatenance anc Service 
coeraticns have been performed by the Maintenance Ccntrol 
Division (HCD) of the Public Works Department (possibly 
Peeeeuidemcn sUclit=1es eed transportation cperaticns). MCD 
mie OeCecr=s Prepare jcb orders for shops personnel and esti- 
mate time and material requirements based upon enginee 


BPomecniomec standards. Inspectcrs periodicaliy visit jo 

piewoce em Vcr. y O,Opie-: Dsz formance of repairs anc mainte- 
bemecw  wieey alsc compaze actual job expenditures with crig- 
inal estimates. As Ch service contracting became fart of 
Navy facilities management, inspection responsibilities have 
been assigned to Maintenance Control Division perscnnel. 
Malet Mt yeEecsS Ci CChitracts inspected include custcdial 
SSsrvices, qrounds maintenance, retuse collection, ate 


Sz 





housing facilities maintenance. Sometimes this responsi- 
Ee ty 2 shared between MCD inspectors and base te2nants 
when the vclume of contracted services increases signifi- 
cantly. This shared responsibility has resulted in somewhat 
merous CA Gentract administration. 

Reliance upen base customers to provide regquized 
surveillance is not always a suitabl2 practice. There is 
always a danger that customers acting in @ contract inspéec- 


eer’ s recle 


@eadhing With the contractor. 


thorized 
disputes 
eee 


Ccustcmsr 


=> 6 > 


Seo t bact 


relief will generally be provided 
changes cr 
Gepends heavily upon customer observa 
not desirable 


assurance precess. 


COon.racs 


Service 


successful CA contracting. 


ge sei nc 


eon. 


Public Werks or Supply Departments. 


ZOcCS in 


skills 


This arrangement 


Increasing 


Gomrteace Manegement 


mee oC. es iy, 


ec that 


Will exerciss implisd or apparen= authority in 
Constructive changes cr wunau- 
lead 


these de not 


changes in scope can =C  COUNTerDroductive 


anemia <igaticn; serve the best inter- 


1lther the government or the contractor Where 
Surveillance methods are used in a way that 
boeton mance standards are wlicimarely raised, hen 

wom eeeCensrac con Under. Aa 


@iepures process. A surveillance system which 


SUOnmOn aa Cone Dac=CrE. is 


i. the PCAesScevece TCenmeract quelity 
{Ref. 42] 


work 


for use 
VY OMuIne,S Mae =a Duca D lS 
of Gente ia 7o0 
age 
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Such organizations are s 


ccnversions make establishment 


OLGabiZan2C1s acvan= 


rate, 
Kev emon OLS On Seuvace 


gee as 
probably be 


The CA ci 


aeson Gav islons, 

and will located 

Sher ace 
ae 
be 


and ma 


such divisicns possess a 


Gl CON = tacu SUL Vez 


Eénhances management 


Beqd@rey san «he contract process. 


nabile, 


ie ae 
working undér one guality assurance 


then 


C2e2ralized organizacion 
Manager 
Optimal organization 


other li¢ss 
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Exist. These are illustrated and explained in Figqu 


through G.4 in Appendix G. These figures show th 
Becmimcy SUPpore Contract Office can administer CA sé 
SGenmtrac=s of a facilities nature, being located és 2 

rate divisicn of the Fublic Works Department (PWD) or 


Subag2viston of the PWEDE Maintenance Control Division. 


may work in the propcsed central organization, or nay 


for functicnal daivisicn managers. [Ref. 45] 


G. The Navy Service Contract Manager 


i 
(p 
in 
Ga) 
e 
as 


A key player in NAVFAC's service contract precess is 


the Service Contract Manager (SCM), the head representative 


cf the activity's quality assurance team. His role is 


described as follows: [Ref. 28] 


pee Sem vecorGoneract Manager is thasc person with dir 
Scoucaveitey  fOf day-to-day mnanagenemnent Oe 
eee SGierect.  Fosae@r tO award 42 iS responsible 
assiscing in the creparation of the statement of wo 
the gcvernment Estimate, . and the surveiliance pl 
Post award r2sponsibi Lities.-ate oo .ehsuce. that 
centract runs smocthiy and is properly managed, < 


tees 


Surveillance is conducted, and documented, that con+ract 


WwOrtkKing files are mainta ined, and that work crders are 
Paopesiy. coordina ted with she Officer-in- ~Charge¢. Tae 
Change crders are required, tha SCM must precess then 
and make a recommendation PO ete “he 
feeedemeretrtCS=- ape Gharge =O issue a change; ir the 


G@meractor =S, having eae the SCM must reccmmend 
en 


seus Boecemeeccecn tO =he ResidentsOrticer-in-Charge 
ctsrs involving quebity, tine, Nconey, Ob ysctecy, 
fist cocrdinate these matters with the eae ear 


< 
ao) 


and 
the 


Comeaac. srecialist, and the Resid¢ent-Officer-in- Charge 


The quality assurance Pedaal provides the Service 
SOuraaGuetenager With information on the conzractor's 
Ben Soe The SCM has t¢chnical and superviscry 
Sobor omea cy £Or this progrtan. 
This andividual may be ezther an Engineering Field 
Division asset or may te on the field activity's rolls, and 
possesses an appropriate level or contract authority. the 


SCM is skilled in tke use of statistical quality assuzan 


O 
(D 


and is able to plan and manage random sampling surveillance. 


Hememeiamiyehas a public relations tole. Navy SCM's may be 
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PeeeneCrE wt atary officers or civilians. POr~e ne sALS eres, 
this playéris known as the Quality Assurance Evaluarozt 
Program Cccrdinator; the major difference between Air Force 
and Navy reles is that the SCM has supervisory respensi- 
bility cver QAE's, while the QAE Program Coordinator ross- 


esses advisory responsibility only. 





5- A Choice ime Coc tsabazed or Decsntralized 
Crganizazions 


Pgmeced ama <he CA service cent 


H 


ac a 
nizaticn, séveral aiternatives may be consider 


meme Cece nhecral:Zsd tc centralized fEormats. The 


Organization has already been discussed; ene Service 
Semtract Manager is completely in charge of this tyre of 
Semucture. Contract specialists and quaiit assurance 


personnel report to the SCM who usually is a division head 
Eepemcina we che activity Public Works Officer or Supply 
CmEi Cer . 


ho 
=o 


At the cther extreme, <th2 SCM is 


ry) 


Ti 
Beeene cattial Supervisory responsibility over insp 
d 


With all quality assurance personnel being assigne 


tional department managers. The SCM may find it difficult to 
Mandate specific detailed procedures to oc fFolicwed ina 
executing the surveillance pregran, with the possible lcss 


of management controi being mest pronounced under this 
scheme. 

Cees Varuat2cnS might iccate the contract ganager 
Meeieme Cte: GaviSions. For 2xam ; he may rteporz te the 


a 2 
Semevicy = Mein venance Control Director. Tnere mav also be 


+-—' 


ete then One SCH manager for an activity. Canal yee! 


Vv 
functional division heads, given the proper un= and level 
Chat rearing, Couva conceivably ob ne 


Managers . 
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PiemeAlL i werce WeTL1sz=s ~h= following Che con@rect 
Management structure. Ali Qualit Assurance Evaluators are¢ 
assigned to functional managers and are assisted in their 
Ger es bywekne OAR Ereqram’ Coordinator, an expert in the 
meet anGe: — SeavaCe CONtEacs surveillance. QAE’s 42 
designated in writing, and a candidate's final appro 
decided by tha installation commander. Training and surveil- 
lance of QAE activities is handied by the QA vrogram cocrdi- 
Macor. Peecat tong Of the qualicy assimance oOrganizaticrn is 
Meeeteraiazed PUESYaAret tO policy stated in ArR 70-9. Some 

Ta 15 


Air Force activities formerly utilized cént 


ry 


mate tegcowm ence authers’ conversation with Air Forc? sources 
revealed that such organizaticns no longer exisx. 

Ne Navy policy exists that mandates either d 
Mee acco a Celcrelized organizations. Aco larger. «2u 
Works Cénters, trends seem to indicate that centralized 
types cf ciganizations manage he ad 
cone Lac=S « Metchin fublic Works Department 


organizaticn may be found. Based on che s 


om FE @ 
9) 
SF 
te 
(n 
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taken, larger Public Works Departments propa 
QAS assets and the Service Contract Manager within tne 


Meeeieenence Control ELivision. 


Gentes tazaticn of  QAE assets 2S 2aeS table" s5o— 
several reasons. One is that effective ccntract nanagenent 
conzrel must be estaklished. Inspectors placed under func- 


tional managers may have little or no ailégiance to main- 
Geen Gcute -Mteqricy of the servic cencract process and 
May net ke able to cenform with standards st py tne SCM. 
Taeae ace 92¢€ Guarantees that Qualiz Assurance Eva 
Web e sd=Gacetea Strictly <o performance of inspecticna. 
Twas, ~evereall integrity of the service contract prec 
ced where the surveillance responsibiiity accrues 
tional division head. Work ordering inputs «fay 
Sanet Erom work cutput verification processes ror 


Pr mGcmurrG-7eleading <o possible contiicts of interest. 
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Seceeaci squdiitie=s of the Service Contract an 


ry 
) 


( 


(b 
4 


were described earlier. His responsibility for grewing 
numbers of A-76 service contracts makes it necessary <o 
appoint tim as a division head. 

The most iiiconcane, FoaSsonmm Loss TO locazt2 QGAE 


perscnnel under functional management to preclude the 


UW) 


Seaavercoaht communication of inval@a implied contracztal 
SaeehOLitywe tc the Serwice cecntractor. Air Force limitaticns 
on QAE rcies are spelled out as follows: {Ref. 38] 


pee S @8Wili not clarify, iieeltret Ge inter Bedqal 
MireemPret actions Of COntract SCOpS Or intent: 

Bb) QAE'sS will not give direction to contractor employses; 

Syechn Ss wil not <nter into unauthorized contract agree- 


ments (including modi fications) ; 


ct 


d) QAE's will not requires work to be done that 1s no 
Spccemcicaliy Galled £cr in the contract; and 
2) QAE'sS will not authorize expanditures of funds. 
These actions are the responsibility of the ccegri- 
Pere GON racCcing agen-=~. In a centralized staffing arz>ange- 
feomegmens ob Can readily identify and ascertain the acticns 
Gmeepen= Quality Assurance Evaluators via daiiy inspection 
BoporTts . Thewocu® will be more successful in providing 
training for QAE's and assisting in their professional self 
develcrement. The Cualic Assuranc2 Evaluators can acre 
BeadtiveaVail themselves of conzract administration assis- 
Boece ciG@eecooperate 0 “establish aintegrizy in the CA 
contract frecess. 
Functional manag2rs may oppose this centraiize 
Mowe GrsielamCnganizgat.¢n concept on the grounds th 
COV mOmoGcmcencrol cf their operaticns. fT 
Sepoldmaeecieove that the quality assurance function is 
Pao heGem-ecronSibiticy o£ their organizaticn, and that ao 
cutside centraiized department shouid interfere with their 


assigned missions 
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Geemetstame- benesihie cssuliting ~iscom a centralized 
axcrangement is eéasier tracking and budgeting for <«he full 
COStS Of service contract administration. Such ccsts may hs 
mete Gifeacujt to identity for decentralized organizaticns 

An aged @£ uncSrtainty in the use of a centralized 


Organization involves job qualifications for QAE's «ho 
Baspect Unigue technical CA functions. In the Public Works 
PexloChm@cn<, this issue might pertain *o inspection cf utril- 
iealc= ana thenSportation equipment sasvice contracts. The 
activity must balance its choice between requirements for 
specialized inspecticn skills and gén2ral knowledge of che 


senoaical area. 


6. GAO 
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GEwem=scent General Accounting Office teperts which 
examined and criticized existing fedaral agency adminiscra- 
mem O= stacilctyeeservice contzacz=s lend credence to tne 
establishment of centralized inspection organizaticns. 

A ete entitled "Bett2z Management Needed in DOD 


co rev 


y 
Zi 


Foaudulent and Exzroneous Contract Payments to 


Reduce 32 


Ki 


al Property Maintenance Costs" of 9 January 1980 


uncovered séveral instances cf overpayments to contractcrs 


{ 


where work was either nor verformed or found to be unsatis- 


Paemomva GAG 9Cinted ©U=x a lack cf effecziv 


(D 
f-- 
| 
(n 
wo 


procedures and internal managemenct controls. anf 

Beeem@es aebe CHt]en Srroneous and unréeéiiable. GAQ called fcr 
independent audits of each SNS We Css Ss perfcraa 
Specific GAC criticisms were: [Ref. 46] 


a) woOrK exzfcrmed was not billed in accordance with 


Q 


Gmieeect PEOV. SIONS; 


b) intericr werk was accepted; 
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c) less expensive maxerials were substituted for <«hcse 
spec. t2edqg in <be Contract: and 

d) some work was paid for more =han once. 

Scmé GAO reccmméendations for improvements were as 
follows: [Ref. 46} 

a) ensure that activities vrovide sufficient numbers of 
adegqudtely Wwrained contract inspectors: 

b) require that rcutine indapendent tests cf Sach in 


f7) 


pec- 
tor's work pe made; 


Cc) 


(W 
+3 
(n 


ure fnat tke proposed work is adequately pianned 
c 


os 


Heme comurac. aWerd and that contract specificaticns 
are clear and arrropriate 
ay COn= 


te lJecal procurement activities; and 


inue to devcte a portion of internal audit effcrt 


@) require that detailed inspection records, including 
measurements and calculations, be maintained in 
cueremeeot COntrace’ payments. 

These recommendations might be better attained with 
a centralized CA contract managmenéent organization. The 
Nee chat aActOnaWdesecs and Space Administration was criticized 
by GAO for certain inadequacies in its ccentract management 
Beeecedures ina 24°O0etober 1980 zeport. Speci ele Cindi ngs 
includeda: {Ref. 47] 


Mim aeGcIioneGtOn Was WOCKing without approved work crdetrs; 
b) guesticnable reimbursements occurred for the contrac 


eae 's WORK: 


c) ccntrect funds were increased before the neéd was 
juseettsc. and 

ee ciesGentsactand OLZLLcSrs had 2 general attitude hat 
Siciwmaollar Value contraccs were not worthy of 
adecuats attention. 


a 





Again, Such problems might pe a 

centralized mcde of contract surveillance and administra- 
aeO T) « Bcth reports emphasize? the importance of prof 
@eganizing ard locating a CA service contract administration 
Organization. The activity may obtain the right numbe 
imspPectors, properly train then, and then gain nothin 
does net jcin these professionals as a unified ¢ O 
handle newly acquired surveillanca 2sponsibilities The 
Northern Divisicn cf the Naval Facilities Engineering 
Command (NORTHDIV) ais investigating alternative s<ructural 
arrangements (devicted in Appendix G) and has asked customer 


activities tc provide opinions pertinent to this ecrganiza- 


tional issue. [ Ref. 45} As stated earlier, no mandatcry 
NAVFAC pelicy has been established. A wide divergence in 
attitudes and motivations at various Navy activities may bs 
meadSsGal tC Setrain frem R®andating a specific policy. 


Immediate dialogue on the issue at DOD policy-making l¢velis 
May help <tc astablish standardized CA inspection organiza- 
250 Guidelines. 


Cther advantages of centralized inspection are mcre 
careful assessments cf propesed changes to current workicad 
£E2ctzeve 


Ss 

Mace rncecased flexibility to adapt to these. More ea 

inspectcr autonomy will allow the QAE co 

ities and choose appropriate surveillance merhods. 
In Gecentralized organizations, inspectors may be 


Semoeemececdein exercising such flexibility and their sched- 


ules may be based on the personal whims cf che functional 
Menager. Che primary cbjective in creating an organizational 
Por mcwets sO G=ant tke inspectors oroper resronsability and 
Gentrel. 
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array c£ Advantages and Disadvantages 9: 


Centraiiza 
pas-Geon che fEraeceding discussion. she ccmrparative 
advantages and disadvantages of &@ centralized organization 
are presented: 
a) Advantages; 
a0) More effective and reliable managemenz ccntrol 


mechanisms can be instituted by the Service 
Con react Manager =o establisn COpel cece 
tee gi 1 ty. 

aos) KiLeqiaace of Quaiic Assurance Evaluators is 


obtained in ensuring that services are delivered 


tty 


re 
as required by the contract. Noms liveion 6 
th 


(0 


inspector motivation results by piacing 
inspector under functional area managers whe may 
not appreciate the nuances of contract surveil- 
aon Gel. 

211i) Ultimate inspection costs may be less with 
centralized organizations. 

zv) Centralized divisions may promct? a greater 
sense of professionalism among Quality Assurance 
Evaluators. Statistical quality assurance tech- 
niques and contract administration frecedures 
ars mastered by each inspector through constant 
cooperation and interface with the SCM. 

v) The contractor's interests are better served by 
Sec onemal 222d Ofganizati on; ieemes “Omega Ue Gs =c Vv 
are more visible and understandable. 

vi) Cppertunitiss for mismanagement and occurtance 

co fraud should be lessened. 


WVeovenrcL eating a Céntlalized oOrganizeticn, the 
Beaune yemloss teadily assigns a DprOopsr pricrity 
*O management of CA contracts and gains a mcre 


a 





Viil) 


Seed ipis= SPGislt tion Whenwieerequssts additic 


2.3 


al 
finding cr training supporc. 

Better opportunities for employee self develcp- 
ment exist. Training can be easily secured and 
Managed. The SCM will pe able to better assess 
the necessary requirements for ¢ach inspector's 


self development. 


b) Disadvantages 


1) 


} 
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iid) 


Mm 
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} 


commands. 


Earlier study discussion examined the feasibils 
—_— ie 


w ccntract agency organization struc 


Gettawn, CA functions Have EeQ Uae = 6Gunicea 1 
expertise that is not available in the existing 
QAE resotrces. By not placing the surveillancs 
for these under functional manager contrel, work 
perfcrmance may be jeopardized. 

Functional manager objections must be dealt 
Wee These? might be a perceived lack of 
control that result from the inability to deal 
With contractors in a face-to-~fac2 relationship. 
They could also feel that centralized inspector 
organizations will be lazgély insensitive to 
special cencerns. The end result of such appre- 
hension may be refusal +0 ccoperate with the 
iMmepectmeneOorganization and ene contractor. 

At small naval activities, ihe Cost! icf . a 
centralized division may exceed zhe cost of 
wlbaceag Cace Inspector under functional division 


heads. 


2 
Whicmrecee 2S A 


c@ centzracts that includes Navy intermediate and field 
h 


Tf these activities eambrace the concept of 


centralized service ccntract divisions, the Navy will have 


moved cne step closez to its creation. 
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Usemonmm Convrals zed Organizations will be a 


geous shculd an activity choose to use large Base Of 


4 
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Beem ert type contract solicitations. Thea activity may 


@esise tC £¢rm quality assurance teans for specific contrac 


c 


pau 


mypes Which exhitit a combination of both decentralized an 
centralized formats. 


C. JCB SERIES DESCRIFTIORS FOR SERVICE CONTRACT MANAGERS 


Mic ecruay has carefully described 4 precess for 
building a successful contract quality assurance crganiza- 
ti0n at «he field level, emphasizing the formaticn of a 
separate division to manage the administration of a grewing 
mimoer CL A-/6 CA service contracts. This occurs after the 
activity defines, estimates, and budgets personnel requirse- 
Mep-s sic Carry out the cerfcrmance of each contract's 
guality assurance plan. This process will be an ongceirg cne 
as incréeasing CA contracting occurs. 

During this process, the activity needs to define and 
Getarmine skill and knowledge requirements for that person 
who may reve to be most crucial in successfully tanaging 
the practical isplerentation of CA service contracts, <h¢ 
S-2vice Centrace® Manager 

Poe Se=ipcaicn cf the Naval Facilities engineering 
Command requirements for this person was presented in the 
Ppecccdmag section. Some additional reguirements of the SCH 


are discussed. 


A gualizty ccntrcl manager must be anole to investigate 
and prepare plans to mest long range quality contrcl needs, 
Paeandeenema zeailistic Objectives. He audits the eifective= 
MeSsS Of in-house quality ccntrol procedures and spet-ckecks 


Gelivered services tc measure quality. He must teé able to 
Bic ze andeancerpoet Lecords resulting from @ quaizty assu- 


hence Ercoran, and ke able to quickly dispose of matters 


Derzaining to defective performancs 


as 





He develops standard procedures for random sg 


a 
PuseGeumer.L>s and for ccllection, tabulation, and reperting 


of 
results tc proper authorities. H@e assistS in surveys of 
guality centrol ‘techrigues utilized by potential suppliers. 
He conducts quality assurance training courses. He maintains 


seqmizance Cr all ctrrent industrial a COLtrol stan~ 
dards, and may research and develop new quality assurance 
technigues. [Ref. 30] 


MesGtmeuciMeG INOFKS Conmercial Activities fail 
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four general categories. Service Contract Managers 


have general knowledge of these caté¢gories which are: 


a) General Housekeeping Services 


i) Custodial services; 
a!) Refuse services; and 
aoe Grounds maintenance services. 


db) Building and Maintenance Sevices: 
) Housing maintenance services;and 
a) Industrial facility maintenance services. 
e) Tr De ion Equipment Operations ard Maintenance 
services; and 
d) Utilities Operations and Maintenance sérvices. 

When functions azé placed under cost study, these will 
indicate the technical and managerial skills to be réguired 
or Semuece Contract Managers and Quality Assurance 
Gomes LOC he =Selvilee 


2eh =ech= 


Evaluators. Possible job series desrip 
Contract Manager will be explored whic 
nical and managerial capabilicies requi 
Bemeraccilery DOSiticn. Specific Ss 

Wage Grade classifications fer Quality Assurance Evaluatctrs 
will net be evaluated in the thesis due to the divessity of 
activity requirements for these positions. 
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The primary consideration in choosing Quality Assutanc:= 
Pyemuccecr= 25 dcterginings if “there will be a paramount 
requirement for actual trade or craft experience. Ii the 
inspecticn duties requires that QAE's be able to perform the 


work, then a Wage Grade classification i Seqducted. 
Otherwise, a General Schedule job description will suffice. 
Civilian personnel offices are most qualified tc deter- 


Mine the classificaticns and grades for QAE billets. It is 


imperative that the activity carefully defin insoection 
WOCTK requirements so that an accurate assessment of specific 
technical and managerial skills can be determined. These 
Will be the basis cf the position description that will be 
used to evaluate the qualifications and prior experience of 
€2a prespvective QAE candidate. The Service Contract Manager 


em@euld be involved inthe initial scresning of such candi- 
dates. 

When the Service Contract Manager role was first defined 
by NAVFAC, it was recommended that this position kre clas- 
eateGg under the GS-1!102 series, Contract Specialist. ees 
series primarily involves the review and control ocver 
GONEracts tc protect the government's interest based ucon 
kusiness, financial, and legal standpoints. This series is 


= 


suitable if the activity desires greater emphais cn main- 
Pingmtne ane ecority of itS Services contracts. It do 


23 0 
in 
i3 
O 
ct 


address technical knecwledge requirements which may be 
Saty tc Manage divesse types of industrial functions. 


Another job series that has been suggested is the GS-809 


series, Sense kuec2en CONLEO] Tnspec coe. Representative 
Biceeon- scammer i= Classification are listed: [fRef. 48] 
a) Reviews of plans and sp#2cifications prior to contract 
Sede cai. ON; 
Dieceeemds Ore-bid and pLre-CcoOnSzruction conterences; 
CMmesibaesvises Conduct of Site surveys; 
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d) assists in dévelopment of specifications 


tomiacosmwath «Gontractors: furnishes eed ome 


(D 
= 


erameccistiuecw2on Schaduling, pregress reporting 
Safety measures, wage and hour law reguirements; 
£) observes and identifies all stages of consctructicn, 
and takes acticn to ccrrect problems; 
g) reviews ccntractor inspection systems and advises of 


necessary corrections; 


<2 


h) investigates and processes chang? orders: 2n 

=) interfaces with local agencies and authorities during 
Conca suc. 1On . 

The GS-809 series places Significant emphasis cn tech- 

nical skill requirements, scm@ empnasis on knowledge of 


contracting skills, and minimal emphasis on managerial carpa- 


Erbitics. This position has been used with success for 
eWo-aViscry COnSstruction inspection positions. De WOU pe 


PSaceesuicatbie for quality assurance specialists or ‘for 


Service Contract Managers who wiil be responsible f 


O 
a 


Surveillance of maintenance construction service contracts. 
Another series is the GS-810 Facilitiés Enginsering 
Manager. imecevsrs duties in the areas cf ltavestigqatiors 
and surveys, planning and désign, construction, research, 
and facilities engineering management. Generally gpeakinrg, 
the GS-310 series does not rerlect a substantial requirement 
For ditective or supervisory control and may not be a mest 
Suitarle chcice. Ht also @acks "a Tequas =men=. £61 Generel 
Mmmwleddesot Cem -ract administration ee ee {Ref. 49] 
Anoczher alternative is the GS-1640, 


A 


-—- = 1 sw 


n 


Manager, which reflects a requirement for proad technical 
Kuowledge CE Operating capabilities and maintenancs reguire- 
Femus EGE dm activilciy’s various types ¢r physical ctilant and 
equipment. Certain specific elements of job performance 


an nngvena 


be? 


include maintenance program planning, financial p 


Comeeeci mand faci ii2ties requirements oblanning. This =post= 
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euem Sm@eemertcoming 1s that it omits @ ré 


guiren 

general knowledge of contract procedures. [Ref. 50] 

All series discussed thus far lack a reguiren 
Oma: 


knowledge inons area that is important f 


development cf CA service centracting, kncowledg £ statis- 
tical quality assurance techniques. This has net been 
menticned in the preceding jcb series although it is 2ssen- 
tial if new sampling inspection systems recommended by A-76 
policy are to be inclemented. 

feeejec series that might best egr wie anees (Ot 
Miaelty assurance, contract administration knewledge, and 


limsated technical and managerial expertise is tha GS-1510 
series, Quality Assurance Inspector. I+ addresses 
design and administration of quality assurance systems that 
invelve menitoring, controlling and maintaining quality and 
tTeliakiiitcy for delivered goods and services. Specific job 
elements inciuds: 


a) Review of the centractor's performance, 

b) réview and acceptance of contractor qualit CONT ECE 
systems 

¢) inspection of delivered services to verify a ccontzrac- 
tor's stated quality cf services, 

GylieecuO® Landon sampling techniques in service contract 
surveillance, 

€) summarization and analysis of resuits of inspecticns 
Of cervices, and 

S\weGeeermit. On OL CCnttactor quality problens 

The Aix Force has utilized tha GS-1910 series 


Quality Assurance Pregram Coordinator positicns at the 
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activity level. mie COOEdINatoL cCondwets traina: 
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functional quality assurance evaluators and aiso a 
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the statement of work and quality assurance plan de 
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processes. He conducts audits of ongoing quality a 
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PacicreseymcoR-rOl procedures for ali Air FPerce (fiti 


ids 


aecavicy ©Chtracts. He may participate in progress meeting 


UV) 


Peaecmom ia ba Wen ECntractors. Working in conjuncticn witha 
Sagm@aezane Centtace administrators as well as the functional 
See se gialtcy Ascsurance Evaluatozt, he 1s a vital part of 
the activity's service contract program. The Navy Sérvice 
Conztract Manager would perform essentially the same rclés, 
except chat this role is expanded to includes direct supervi- 
pepe cL CAE" Ss. 

Use cf tne GS-1910 saries is recommended as the mest 
Suitable pesition descripticn for Services Contract Manacers. 
Pieuetonel | thadining an contract administration crecedures 
May ke required, but this can bé easily accomodated by 
aoemmmecoe SeM tO a contracting scheol (such as the Civil 
BMganeecr Ccrts Officers School at Port Hueneme, California.) 


Meas SOSition iS operational in nature, and requir 


f 


a 
working kncewledg2 of various statistical Pe assurance 
teckhnigues. zt is urlikely any of the previously mentioned 
job series will enable the activity to secure a manager with 
this specialized background. An industrial engineering kack- 
ground for prospective SCM candidates is also recommended. 
The rest of the inspection organization can supplement 
and augment the general knowledge held by the Service 
Contract Manager. Cuality Assurance Evaluators should 
Possess special skilis in carpentry, utility equipment, 
ReanSeortaticn vehicle equirment and other specializec skili 
zs2equirements. Based on preceding discussions of organiza- 


ués, the hiring of ccentract speciaiists may be 
= 


f +> 


“ioral 
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HeGecs a 


ct 


- Such e¢kilil and technical knowledge require- 
Serco stem be antluenced by the size Of a naval activity, 


2ts ccemplexity, and the number of service contracts that are 


being performed. ie mey be necessary co hire statis*ieal 
specialists at either intermediate or top level commands so 
that ongcing review and improvement of statistical surveil- 


Wemliee Metreccs Can be carried cu. 
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Pee victicsS@must acclUrately decerwine the qualificaticns 
Mencune Sevice Combract Manager, especially if they elect ®o 
form a centralized service contract management organizaticn. 
Paesdrality £6 successfully execute Commercial ‘Activities 


Vere t2eq co che skills of the SCM and his corgenwzatdon. 


D. SUMMARY 


This charter has examined severai issues that relate to 
Been ctreetuOn OL an inspection organization fer these service 
Comet tracts that result from OMB Circular A~76 implementation. 
The Nav 


a 
y field activity that receives services has the msajorz 
Beem ewsslhacy LCL Ensuring proper contract administratio 
by previdina on-site surveillance resources. 

The acegveey aust fellow a sequence of actions that lead 
merene G2sce<ciOn Of an onsite inspection crganizaticna ace 
Must first estimate hcw many inspectors it wili need, sale 
Peete cGeret> additional inspsctors or utilize existan 
rersennel zesources. 

It must ensure that a@ training program is estanlished 

PEOGC 


woich zducetes the inspectors in qua 
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Q «tl 
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Guz2es and furthers theiz knowledge in n inistra- 

Belemecieamesrileos. It gust secure funding for this effert. 
The achivic Bust Gheoose the GOrganizat=on Lemmas ra 
est SUitsSs its surveillance needs. Althcugh reséarch 
eugeests that 2 centralized format is prefetarle, “he 
qomeiyity Shey Elect tc place inspectors under functioral 
Managers. Either system must Si¢> Sutticisac ganaccmen* 
isned. 


= ° u e 
Soemcl eto w=aSise thac contractual integrity is estarbil 
os 


Popeye, assed ULCH the. cype of contrac 


ah adCtivity must define knowledge and skill Tequirements for 
SaeweGletesy “ASSurance Svaluatcrts jeg. SWirvace Cepzeac= 
Pesadercseemne SCM Decsition is vi~zally imporcant, and 2 mcst 


Capammc Qoctvidual with training in statistical quality} 


contrel techniques is required. 
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WeteGNPE CO Per  Lntegration of each of these activities, 
Navy activities can expect tec be in control of the evelutvicn 


Bede crores 2iplementaticn of OMB Circular A-76. 
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Aw. ISPROVEMENTS IN STATISTICAL QUALITY ASSURANCE 


It is evident that quality assurance techniques are 


Meaecul ang kbteneficial in the CA service contract process, 


srecmoting fairness and equity *o all concerned parties when 
used properly. These techniques should ultimately reduce 
mem@eecact €CSstS te the government and contractors, anda lsad 
tO mere réliabie assessment and documentation of service 
Semitact guelicy. Statistical Otgiveewecontz cl (SOC) apple 
Cations wili be limited only by the creativity and imagina- 
tion cf those who use then. 

AeCCOBGIAGLY, tne rollowing recommendations for the 
quality assurance precess are offered.A formal working ranel 


ef DOD cfficitals should b2 organized to review the different 
Statistical quality assurance methods ‘that are either being 
Utilized cr béing studied. AECCUTATS, Mere liable 19tfcrmazden 
that examines the merits and weaknesses of acceptance and 
estimaticn sampling frocesses should be made availadie to 
this rvanéel. Perhaps an independent, unbiesed, quel: 
contrel expexrt should be included on this panel ‘toc ensure 
that ¢ach member properly understands the validity and 
application of statistical guality assurance tc CA service 
Sentracte. The authcrs believe that any confusion amcng 
Wat2ous DOD CA manacement officiais concerning SQC *echni- 
ques can be eliminated by properly and accurately stating 
Winch Cir wuaese Can be used in ongolag A-76 cohtracts. 

A ccost-rcenefit and risk analysis may b# conducted to 
examine which of th2 two sampling processes are more appro- 


priate fcr CA quality assurance. Paxed ) budgetary “funding 


et 





amounts fcr CA ccntract administration may have significant 
influence in the determinaticn of which standard sveren 


should bre adopted. 


Bo IMPRCVEMENTS IN ADMINISTERING EXTRAPOLATED DEDUCTIONS 


A unifcrm policy should also ba adopted for the extrapo- 
lated deductions process. As was observed earlier, the Air 
Force dces not deduct for defacts when these are iess than 
the reject number at the specified acceptabie quality level 
ony. New NAVFAC pclicy, if adopted, would cali for deduc- 
tions for ail observed defects and liquidated damages set as 
a percentage of the value of Gach observed defective work 
item. Current Air Ferce and proposed NAVFAC policy would 
implement extrapolated deductions from sampies to Icts when 
specified ACL's are exceeded. 

The authors believe that the NAVFAC policy should be 
adopted. Itis more realistic, ozoviding more enforcement 
power in the long range evolution of CA Service contracting. 
It may also be more defensible in subsequent GAO and cther 
DOD audits of each agency's CA contract administration 
eELOCtS. The previously mentioned DOD A-76 workina fare 
that studies sampling techniques should be tasked to resolve 
Mees divergence in administraticn volicy. 

The authors reccmmend an immediate resolution of this 
meicwuceallLew tne presentation to itS contractor ccnmunic 
mmo ne Ueenceum DOD CA ~volicy concerning contract deductions 
Mui soeCauod. failiure co do So will hinder optimal pesécr- 
Mance cf gcvernment surveillance, and possibly cause it to 


become ineffective. 
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Cw. IMPROVEMENTS IN ESTIMATING INSPECTION RESOURCES 


[iemteret MajOE issue that waS Covered in this <«ofic 
O° 


area was the preper estimation of contract adminis 


ct 
rf 
oo 
ct 
e 


resources, particularly the determination of the required 
number cf Quality Assurance Evaluators. Pied. peiae 
adopted that ttempots to correlate the number o 
inspectors with the rumber of contracts, their 4 

or the number of positions béing converted, then a ccmpre- 
hensive study shculd be undertaken 9f A-76 resource expendi- 
meses Dy all federal agencies and activities +0 facilitate 
mee dgevelcprent of an estimating model. All facets cf CA 
service ccntract administration resources consumed for the 
past five tc ten years will need to be carefully examined; 
such a Large inclusive data base will be the mest reliacle 
méans of Euilding this model; however, the cost and time 
requirements for its development may b2 prohibditive. 

The autnors recommend a much simpler and more accurate 
approach Similar to that which was developed by the Atlantic 
Divisicn, Naval Facilities Engineering Command. This model, 
when used properly, should provide a relatively unbiased and 
unccnstrained estimate of resource reguirements. If this 
technique is used, sincere attempts must be made by all 
federal agencies to procure and place the required inspec- 
tion rescurces at each field activity. N Cape nls2n eegra— 
meen Of A=76 resource planning activities in each agency's 


annual -tudcet precess will be required. 


Mmmaicor m= 2za=cn OL Sati Sf£icing occur ir <his frocess, 
men the first, more detailed, model shouid be developed 
EamtGo is Will prcbably be more indicative of éctual inspéc- 

on 


requirements. 
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D. ENHANCEMENT OF CURRENT QAE TRAINING 


During the course of the study reseatch, She -atencr 
cbserved differen training programs for Quality Assurance 
Evaluators. In general, these were sufficient in previding 
Mmieetal €XECSure to rudiments of contract law, sampling 
techniqués, and other skill areas required of QAE's. Tnese 
kasic training courses should be enhanced by instituting 
Mere Cngecine activity training programs which #ither rsit- 
erate or augment the fundamenta precepts of CA seéerv 
Seneract admianastraticn and, in turn, lead to a more prefes- 
eaonal QAE staif. 

The Air Force has addressed this need in desi 
QAE training programs. The QA Precgram Coordinator 
peGoOtcanuang dialoque with cach QAE during the pe 
Derwods OL Service ccntracts, WleneCOntL aca team? 
also ciftering their advice and assistance. 


Mie Navy has adopted the practice o£ offsting ztegional- 


ized training EOEMPOR ES ac welhtber Engineering Field 
Divisicns, large Puktlic Works Departments, or Public Works 
Centers. QAE candidates travel to these sites and spend a 
week in training. Few cngcing training pregrams at the 


activity level were ckserved. 

The authors reccmmend that existing agency efforts in 
*he training phase of CA implementation be ent imwed. 
Imprcevemente shcsuld te effected in creating more ongoing 


Geeumaey cLaining prcgrams, perhaps adopting a prcgrammed 


j de 
FP | 


(A 
ct 
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eGemeedeey ce Of 2=nSstruction technique. Training in 


m 


tical quality assurances can be improved further, wit 
greater emphasis placed on basic statistical theory aad 
aepkecaticn « 

Pendenguctwe.« LOr OAR training is crucial to emakisc 
the quality assurarceé programs for CA sérvice contracts to 


ke successful. Haphazard and poorly plarned funding will 
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jeopardize the enrichment of skills of each QAE and peossitly 


detract from the successful evolution of the A-76 percegzan. 


me FACILITATING CA ADSINISTRATION CONTROL THROUGH 
CENTRALIZATION 


A large portion of the study addressed <th2 creaticn of 
Se 7elaiizead CA contract administration organizations. It is 
realized that this crganizaticnal form will not be actpro- 
priate fcr all DOD agenciesS and activities given 2 wiie 
diversity in size and missicn requirements; many < 
may Ce reluctant to embrace this formazt ev € 
proven to be feasible and appropriate. Earlier discussion, 
however, indicated that the advantages of centraliz C 
outweigh the disadvantages. 

This structure also has the advantages of qaining immed- 
jate manasement contzrcl and integrity of CA contract admin- 
istration in the current developmental stages cf A-76 
lmpilementaticn. Congressional oversight and public opinion 
faite be VELY Cricvical of ccst overruns aad contract irregu- 
jlarities, eéspecially if a large number of displaced iederai 
emplcyees vcice their objections. Decentralized contrel is 
not guaranteed to result in such abuses, but it would invits 
eaescr cecurerce more readily than will centralized inspec- 
mon OlLdganziacicons.Centralized inspection will more readily 
Bec2licate coordinated inspector e2fforts than decentralized 
rormats. Me eErsditilitcy CE inspection Sfitorts may be 
Semreas wich centralized organi zatiors. 

melemeare<=a5OnS for SUDDOITing créeticn of centzal- 


mdattgantons are that full costs of inspection may de 


m 
Oo Oo &@ A 


d, training pregrams may be more effectively adminis- 

tered, and lires of communication from government represen~ 

veS to contract administrators will be made more clear. 

The authers recommend that the centralized organization 
u 


Structure be adopted. 
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Fe SERVICE CONTRACT MANAGZR QUALIFICATIONS 


The Service Contract Manag2zr will be a kev accivity 
individual whether centralized or decentralized ozganiza- 
tions aze chosen. He should be well versed in statistical 
Sampling and have a broad general background in contract 
aGministraéticn principles, as well as having some knowledge 


emmeene furceional areas to be contracted out. Misa eer s 


recommend use of the GS-1910 seriés, as 2 (Cfecr> tHe Nese 
vetsitality and mlexi pi lity in ©Vers=sing CA sérvice 
contract administraticn. Other job dascriptions were exan- 


ined, and in fact, scme of these have actually been utilize 
by various Navy activities. For examples, the GS-1102 series 
could be utilized despite its lack of technical knewledge 
requiremerts. Th¢ overrriding necessity for knowledge of SQC 
Techniques and their applications should licad to unifern 
peeeccence CL the GS$-1910 series as a CA contract adminis- 


traticn manager stancard. 


G. FUTURE CA QUALITY ASSURANCE IMPROVEMENTS 


In its Command Management Guidance for fiscai years 1984 
to 1990, NAVFAC hsadquarters stresses the attainment of 
Mest efficient organization" (MEO) SyEuUCtUre by all of acs 


activities. This thesis examined initiatives in the deveicp- 
Ment cf quality assurance programs which should lead to such 
cost eccnemies while achieving reguired levels of service. 
These should also improve the prefessionalism exhibited by 
sderal acencies in executin further Commercial Activities 
contracts. 

Ccmprehensive pianning efforcs should be 1 a 
all Navy crganizaticn levels to develop an effectiv 
implementation strategy, and integrat2 contract design with 
quality assurance. ther Navy Systems Commands bssides 

D 


NAVFAC are responsible for 4 substantial num 
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Commercial Activities subject to OMB Circular A-76. These 
commands shculd follcw NAVFAC's lead in executing their cwn 
CA programs. 

Continued pursuit of the A-76 policy will change the 
character cf mission performance. DOD agencies will need to 
car2fuliy plan fer their administration of service contract 
programs tc preclude deterioration in the quality of the 


services they receive. 
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APPENDIX A 
LEGISLATIVE FROVISIONS 


Peemcclt OWING ate 2xcerpts from <wo Public Laws thet 
establish congressicnal policy and recurring trestricticns 
concerning ths conversion of DOD In-house activities to 


BemnLact Eexrfornancs. 





Peooeem@emeo. Defense Appropriation Authorizaczon Act, 1975, 
mipbie pew 93-365, August 5, 1974 


SCs S02. It is the sense of Congress < 
Department of Defense shall use the least costly 
Manhpower that is censistent with military requi 
and cther needs of the Department of De 
Peaetefcre, in develc ze the annual manpower authe 
mom SegGuestsSs t¢O the Congress and in carryin 
Manpower pclicies, the Secretary or Defense shall, << 
Partlcular Gonsider tno advanzaqes of Converting £20 
ome form ¢c ne RE OGE 1c ano eDerw (tili taryeece. Villian, “oO: 
Pee Vace CCh_ract) feG@Gecwe DPeCuLOr mance Ofwea Spec2ticd 
Ones este” gilewer tl cat ton Of any CCnhvers2on [rem che 
C@h OL B@anpeser tc another shali be contained in the 
annual manpower requirements tepert +0 the Congress 
aS Meeea CY section 138(c) (3) of titie 10, United States 
ode. 


Department cf Defense Authorization Act, 1381, Public Law 
96-342, SsEiember 38, 1980 
Sec. 2022 
Qyentemecwme=e72a!]° Of industrial type functior of the 
Eépartment of Defense haz on OCtober 1, 1980, is 
Eeing performed by Pepartment of Defense personnel 
May be converted tc perfcrmance ny a4 private 
Generac. CL—— 
1) tc..86 circumvent any Civilian personnel 
Ce miing; OF 
1i) unless the Secretary of Defense provides tc 


the Congress in a timely manner-- 
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c) 


C) 


dq) 


Pow leet lCcateon sor any decision to study 
Such commercial or industrial type fuzc- 
ceo LOS ~POSSib! € performance by a 
Private Cen eeaccor ¢ 


2. a detailed summary of 2 ccmpariscn cf 
muer conc, Of perrormane> of such funct 209 
by Cepartment or Defense personnel ana 
ei RE Vece COneraceo= when demonstzates 
that the performance of such function by 
Pero Vace es GOontlaccor Will. Ereeule if 
cost savings to the Government over the 
IGG sOt the SGOntsact sand 4g Cebe=©. cation 


Ebae the entare cost comparison 22S avai- 


$v 


lable; 

eee Celt itacat.on fhat the Government 
CalewhasLOR £95 th] Cost ef pertcrmance 
of such EUNGE LO he ayy Department 9 
Defense personnel is based cn an esti-~ 
Mate oft ths most efficient and ccst 


SLicaerve OLgamezdtsOn fOr (perternance 
o£ such SuUnGr On. soy Devas ? 
Defense personnel; and 


4. a =report,, to be submitted with ths 
certification required by subparagraph 
SNOW ING==— the petenrial | scononme 

Ffect on employees aftrected, and the 

potential economic effect on the lccal 


D WWJ 


Community and Federal Government if mere 
than 50. employees are inveived, er 
GOntracti ng EOr PpSEroanance cf such 
MUMGeOn. GAC CLrCe:  Ormecenerac.indg for 
Pobeoemance OLN SUCH “Lume<10n On the 
Military mission of such function; and 
the amount of the bid accepted for the 
performance of such fEunction by the 


private contractor whos? bid is accepted 
Andee COSt 16: pcr Mommance Cr Telen 
fu ne-.0n by Department Gu Defense 
personnel, together with costs and 
expenditures which the Government wili 
maewGE beCaus= Of Pho Conerace. 


Mmemearter ccipleticn of the studies required for 
ecemelet ion or the Gem ceer tear ec lye D report 
required by subparagraphs 3 and 4 of subsection 

meee GecoSlOn 1S made €O CONVSEt EO Contrac zor 
errormance, tne Secretary or Defense shall notify 
engress of such decision. 


j-4- 


ihe Sacretar of Defense shall submit a wer 
report to the Congress by February 1 cf 
mea S au eee GesCripbm@ng, ene Sxtetee vO ewer CC 
Gea wana andustrial type tuncecions were pert 
ty Department of Defense contractors curing 
preceding fiscal year. The Secretary 
goely@e2 ah ¢ach such report ah estimate or 
ercentage of commercial and industrial ctype. 
cicns of the Department of Defense that wi 
performed by Cepartment of Defense personnel 
she percentage cf such functions that wil 
erftormed Dy Daiviaste —eCchunectOr Sc, e Oucin £ 
iscal ysar during which the report is submitted 


Os 


in 
too 6ODlUWN Bact 


Sit ety dD Mct 


IMDM fu st 


e 
rh 


fy 
raed VOTO awd 


DMD Cum 


ime section shail take esfHec= on October 1, 1980. 
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APPENDIX B 
EXCERPTS FROM SMIL~-STD-105D AND MIL-HNDBK-53-18 


ieee cep PD Sndix, informaticn contained in both Malitary 
Standard 105D and Military Handbook 53-1A is presented. The 
x 


exe ce NEL=-STD-105D is given in pages 111 through 120. Key 
Pree omees CONS aGsreqd in she uss of the standard are <che 


€, the inspection intensity, and the desir2ad accep- 
ality level (AQL). A déterminaticn of these varia- 
results in a sampling size and accept/reject rumters. 
Gating characteristic (0C) curves for varicus sampling 
€é also presented in pages 121 through 127. These 
Mpmecehc ene ~ExrCcection that 1s offered t0 Ecth the 
@encteactOc and the “government, being 4 function of the 


percent cf defectives that are found in samplss. 


th 


ae 


5 


Fagss Wjioue = ns ough 131 are selected pages 
MIL~HNDBK~S3-1A which provide mors Gasetailed explanations of 


it 


ome OPerat= ng characteristic curves. Page 132 is an €xc¢erIp 
feom a Zandcm number table. Finally, pages 133 and 134 are a 
fieescang Of key activities tc be Eollowed in implementing 
Piee-SotD-1050. 
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SAMPLING PROCEDURES AND TABLES 
FOR INSPECTION BY ATTRIBUTES 


XD 


he 


1.1 PURPOSE. This publication estab- 
lishes sampling plans and procedures for 
inspection by attributes. When specified by 
the responsible authority, this publication 
shall be referenced in the specification, con- 
tract, inspection instructions, or other docu- 
ments and the provisions set forth herein 
shall govern. The “responsible authority” 
shall be designated in one of the above 
documents. 


1.2 APPLICATION. Sampling plans des- 
ignated in this publication are applicable, but 
not limited, te inspection of the following: 


a. End items. 
b. Components and raw materials. 
c. Operations. 
d. Materials in process. 
e. Supplies in storage. 
f. Maintenance operations. 
Data or records. 
h. Administrative procedures. 


These plans are intended primarily to be 
used for a continuing series of lots or batches. 


SCOPE 


The plans may also be used for the inspection 
of isolated lots or batches, but, in this latter 
case, the user is cautioned to consult the 
operating characteristic curves to find a plan 
whicli will yield the desired protection (see 
11.6). 


1.3. INSPECTION. Inspection is the proc- 
ess of measuring, examining, testing, or 
otherwise comparing the unit of product (see 
1.5) with the requirements. 


1.4 INSPECTION BY ATTRIBUTES. I[n- 
spection by attributes is inspection whereby 
either the unit of product is classified simply 
as defective or nondefective, or the number 
of defects in the unit of product is counted, 
with resnect to a given requirement or set 
of requirements. 


1.5 UNIT CF PRODUCT. The unit of 
product is the thing inspected in order to 
determine its classification as defective or 
nondefective or to count the number of de- 
fects. It may be a single article, a pair, a set, 
a length, an area, an operation, a volume, a 
component of an end product, or the end 
product itself. The unit of product may or 
may not be the same as the unit of purchase. 
supply, production, or shipment. 


ipl 





2. CLASSIFICATION OF DEFECTS AND DEFECTIVES 


2.1 METHOD OF CLASSIFYING OEFECTS. 
A classification of defects is the enumeration 
of possible defects of the unit of product 
classified according to their seriousness. A 
defect is any nonconformance of the unit of 
product with specified requirements. Defects 
will normally be grouped into one or more 
of the following classes; however, defects 
may be grouped into other classes, or into 
subclasses within these classes. 


2.1.1 CRITICAL DEFECT. A critical de- 
fect is a defect tha. judgment and experience 
indicate is likely to resuit in hazardous or 
unsafe conditions for individuals using, 
maintaining, or depending upon the product; 
_or a defect that judgment and experience 
indicate is likely to prevent performance of 
the tactical function of a major end item such 
as a ship, aircraft, tank, missile or space 
vehicle. NOTE: For a special provision re- 
lating to critical defects, see 6.3. 


2.1.2 MAJOR DEFECT. A major defect 
is a defect, other than critical, that is likely 
to result in failure, or to reduce materially 
the usability of the unit of product for its 
intended purpose. 


2.1.3 MINOR DEFECT. A minor defect 
is a defect that is not likely to reduce ma- 
terially the usability of the unit of product 
for its intended purpose, or is a departure 
from established standards having little bear- 
ing on the effective use or operation of the 
unit. 


2.2 METHOD OF CLASSIFYING OEFEC- 
TIVES. A defective is a unit of product which 
contains one or more defects. Defectives will 
usually be classified as follows: 


2.2.1 CRITICAL OEFECTIVE. A critical 
defective contains one or more critical de- 
fects and may also contain major and or 
minor defects. NOTE: For a special provi- 
sion relating to critical defectives, see 6.3. 


2.2.2 MAJOR DEFECTIVE. A major de- 
fective contains one or more major detects, 
and may also contain minor defects but con- 
tains no critical defect. 


2.2.3 MINOR DEFECTIVE. A minor de- 
fective contains one or more minor defects 
but contains no critical or major defect. 


3. PERCENT DEFECTIVE AND DEFECTS PER HUNDRED UNITS 


3.1 EXPRESSION OF NONCONFORM. 
ANCE. The extent of nonconformaince of 
product shall be expressed either in terms 
of percent defective or in terms of defects per 
hundred units. 


3.2. PERCENT DEFECTIVE. The percent 
defective of any given quantity of units of 
product is one hunderd times the number of 
defective units of product contained therein 
divided by the total number of units of prod- 
uct, 1.e.: 


i 


p feee Number of defectives 
ercent celcch’ <=) Numborsomunits inspected 


3.3 DEFECTS PER HUNDRED UNITS. The 
number of defects per hundred units of any 


* 100 


given quantity of units of product is one 
hundred times the number of defects con- 
tained therein (one or more defects being 
possible in any unit of product) divided by 
the total number of units of product, 1.e.: 


Number of defects 
= “Number of units inspected 


Defects per 


x 100 
hundred units 








4. ACCEPTABLE QUALITY LEVEL (AQL) 


4.) USE. The AQL, together with the 
Sample’ Size Code Letter, is used for index- 
ing the sampling plans provided herein. 


4.2 DEFINITION. The AQL is the max- 
imum percent defective (or the maximum 
number of defects per hundred units) that. 
for purposes of sampling mspection, can be 
considered satisfactory as a process average 


(see 11.2). 


4.3 NOTE ON THE MEANING OF AQL. 
When a consumer designates some specific 
value of AQL for a certain defect or group 
of defects, he indicates to the supplier that 
his (the consumer’s) acceptance sampling 
plan will accept the great majority of the lots 
or batches that the supplier submits, pro- 
vided the process average level of percent 
defective (or defects per hundred units) in 
these lots or batches be no greater than the 
designated value of AQL. Thus, the AQL 
is a designated value of percent defective (or 
defects per htindred units) that the consumer 
indicates will be accepted most of the time 
by the acceptance sampling procedure to be 
used. The sampling plans provided herein 
are so arranged that the probability of ac- 
ceptance at the designated AQL value de- 
pends upcn the sample size, being generally 
higher for large samples than for small ones, 
tor a given AQL. The AQL alone does not 


5. SUBMISSION 


5.1 LOT OR BATCH. The term lot or 
batch shall mean “inspection lot” or ‘‘inspec- 
tion batch,” i.e., a collection of units of prod- 
uct from which a sample is to be drawn and 
inspected to determine conformance with the 


acceptability criteria, and may differ from a 
collection of units designated as a lot or batch 


+? 


describe the protection to the consumer for 
individual lots or batches but more directly 
relates to what might be expected from a 
series of lots or batches, provided the steps 
indicated in this publication are taken. It is 
necessary to refer to the operating character- 
istic curve of the plan, to determine what 
protection the consumer wil! have. 


4.4 LUMITATION. The designation of an 
AQL shall not imply that the supplier has 
the right to supply knowingly any defective 
unit of product. 


4.5 SPECIFYING AQLs. The AQL to be 
used will be desiznated in the contract or by 
the responsible authority. Different AQLs 
may be designated for groups of defects con- 
sidered collectively, or for individual defects. 
An AQL for a group of defects may he des- 
ignated in addition to AQLs for individual 
defects, or subgroups, within that group. 
AQL values of 10.0 or less may be expressed 
either in percent defective or in defects per 
hundred units; those over 10.0 shall be ex- 
pressed in defects per hundred units only. 


4.6 PREFERRED AQLs. The values of 
AQLs given in these tables are known as 
preferred AQLs. If, for any product, an AQL 
be designated other than a preferred AQL, 
tnese tabies are not applicable. 


OF PRODUCT 


for other purposes (e.g., production, ship- 
ment, etc.). 


5.2 FORMATION OF LOTS OR BATCHES. 
The product shall be assembled into identi- 
fable lots, sublots, batches. or in such other 
manner as may be prescribed (see 5.4). Each 
lot or batch shall, as far as is practicable, 


i) 





5, SUBMISSION OF PRODUCT (Continued) 


consist of units of product of a single type, 
grade, class, size, and composition, manu- 
factured under essentially the same condi- 
tions, and at essentially the same time. 


5.3 LOT OR BATCH SIZE. The lot or 
batch size is the number of units of product 
in a lot or batch. 


5.4 PRESENTATION OF LOTS OR 
BATCHES. The formation of the lots or 


batches, lot or batch size. and the manner 
in which each lot or batch is to be presented 
and identified by the supplier shall be des- 
ignated or approved by the responsible au- 
thority. As necessary, the supplier shall 
provide adequate and suitable storage space 
for each lot or batch, equipment needed for 
proper identification and presentation, and 
personnel for all handling of product re- 
quired for drawing of samples. 


6. ACCEPTANCE AND REJECTION 


61 ACCEPTABILITY OF LOTS OR 
BATCHES. Acceptability of au lot or batch 
will be determined by the use of a sampling 


plan or plans associated with the designated 
AQL or AQLs. 


6.2 DEFECTIVE UNITS. The right is re- 
served to reject any unit of product found 
defective during inspection whether that 
unit of product forms part of a sample or 
not, and whether the lot or batch as a whole 
is accepted or rejected. Rejected units may 
be repaired or corrected and resubmitted for 
inspection with the approval of, and in the 
manner specified by, the responsible au- 
thority. 


6.3 SPECIAL RESERVATION FOR CRITI- 
CAL DEFECTS. The supplier may be required 
at the discretion of the responsible authority 
to inspect every unit of the lot or batch for 


critical defects. The right is reserved ts in- 
spect every unit submitted by the supplier for 
critical defects, and to reject the lot or batch 
immediately, when a critical defect is found. 
The right is reserved also to sample, for crit- 
ical defects, every lot or batch submitted by 
the supplier and to reject any lot or batch 
if a sample drawn therefrom is found to con- 
tain one or more critical defects. 


6.4 RESUBMITTED LOTS OR BATCHES. 
Lots or batches found unacceptable shall be 
resubmitted for reinspection only after all 
units are re-examined or retested and all de- 
fective units are removed or defects cor- 
rected. The responsible authority shall deter- 
mine whether normal or tightened inspection 
shall be used, and whether reinspection shall 
include all types or classes of defects or for 
the particular types or classes of defects 
which caused initial rejection. 


7. DRAWING OF’ SAMPLES 


7.1 SAMPLE. A sample consists of one 
or more units of product drawn from a lot or 
batch, the units of the sample being selected 
at random without regard to their quality. 
The number of units of product in the sample 
is the sample size. 


7.2 REPRESENTATIVE SAMPLING. When 
appropriate, the number of units in the sam- 
ple shall be selected in proportion to the size 
of sublots or subbatches, or parts of the lot or 


batch, identified by some rational criterion. 





7, DRAWING OF SAMPLES (Continued) 


When representative sampling is used, the 
units from each part of the lot or oatch shall 
be selected at random. 


7.3. TIME OF SA*APLING. Samples may 
be drawn after all the units comprising the 
lot or batch have been assembled, or sam- 


ples may be drawn during assembly o° the 
lot or batch. 


7.4 DOUBLE OR MULTIPLE SAMPLING. 
When double or multiple sampling is to be 
used, each sample shall be selected over the 
entire lot or batch. 


8. NORMAL, TIGHTENED AND REDUCED INSPECTION 


8.1 INITIATION OF INSPECTION. Nor- 
mal inspection will be used at the start of 
inspection unless otherwise directed by the 
responsible authority. 


8.2 CONTINUATION OF INSPECTION. 
Normal, tightened or reduced inspection 
shall continue unchanged for each class of 
defects or defectives on successive lots or 
batchs except where the switching proce- 
dures given below require change. The 
switching procedures given below require a 
change. The switching procedures shall be 


applied to each class of defects or defe-tives, 


independently. 
8.3 SWITCHING PROCEDURES. 


8.3.1 NORMAL TO TIGHTENED. When 
normal inspection is in effect, tightened in- 
spection shall be instituted when 2 out of 5 
consecutive lots or batches have been re- 
iected on original inspection (i.e., ignoring 
resubmitted lots or batches for this proce- 
dure). 


8.3.2 TIGHTENED TO NORMAL. When 
tightened inspection is in effect, normal in- 
spection shall be instituted when 5 consecu- 
tive lots or batches have been considered 
acceptable on original inspection. 


8.3.3 NORMAL TO REDUCED. When 
normal inspection is in effect, reduced ins pec- 
tion shall be instituted providing that all of 
the following conditions are satisfied: 


a. The preceding 10 lots or batches (or 
more, as indicated by the note to Table VIII) 
have been on normal inspection and none 
has been rejected on original inspection; and 


b. The total number of defectives (or de- 
fects) in the samples from the preceding 10 
lots or batches (or such other number as was 
used for condition ‘‘a” above) is equal to or 
less than the applicable number given in 
Table VIII. If double or multiple sampling 
is in use, all samples inspected should be in- 
cluded, not ‘'first” samples only; and 


c. Production is at a steady rate; and 


d. Reduced inspection is considered de- 
sirable by the responsible authority. 


8.3.4 REDUCED TO NORMAL. When re- 
duced inspection is in effect, normal inspec- 
tion shall be instituted if any of the following 
occur on original inspection: 


a. A lot or batch is rejected; or 


b. A lot or batch is considered acceptable 
under the procedures of 10.1.4; or 


c. Production becomes irregular or de- 
layed; or 


d. Other conditions warrant that normal 
inspection shall be instituted. 


8.4 DISCONTINUATION OF INSPECTION. 
In tHe event that 10 consecutive lots or 
batches remain on tightened inspection (or 
such other humber as may be designated by 
the responsible authority), inspection under 
the provisions of this document should be 
discontinued pending action to imnprove the 
quality of submitted material. 
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9. SAMPLING PLANS 


9.1 SAMPLING PLAN. A sampling plan 
indicates the number of units of product 
from each lot or batch which are to be in- 
spected (sample size or series of sample 
sizes) and the criteria for determining the 
acceptability of the lot or batch (acceptance 
and rejection numbers). 


9.2 INSPECTION LEVEL. The inspection 
level determines the relationship between 
the lot or batch size and the sample size. The 
inspection level to be used for any particular 
requirement will be prescribed by the re- 
sponsible authority. Three inspection levels: 
I, II, and III, are given in Table I for general 
use. Unless otherwise specified, Inspection 
Level II will be used. However, Inspection 
Level I may be specified when less discrimi- 
nation is needed, or Level III may be speci- 
fied for greater discrimination. Four addi- 
tional special levels: S-1, S-2, S-3 and S—4, 
are given in the same table and may be used 
where relatively small sample sizes are neces- 
sary and large sampling risks can or must be 
tolerated. 


NOTE: In the designation of inspection 
levels S-1 to S—4, care must be exercised to 
avoid AQLs inconsistent with these inspec- 

tion levels. 


9.3 CODE LETTERS. Sample sizes are 
designated by code letters. Table I shall be 
used to find the applicable code letter for the 
particular lot or batch size and the prescribed 
inspection level. 


9.4 OBTAINING SAMPLING PLAN. The 
AQL and the code letter shall be used to ob- 


eR 


tain the sampling plan from Tables II, III or 
IV. When no sampling plan is available for a 
given combination of AQL and code letter, 
the tables direct the user to a different letter. 
The sample size to be used is given by the 
new code letter not by the original letter. If 
this procedure leads to different sample sizes 
for different classes of defects, the code letter 
corresponding to the largest sample size de- 
rived may be used for all classes of defects 
when designated or approved by the respon- 
sible authority. As an alternative to a single 
sampling plan with an acceptance number 
of 0, the plan with an acceptance number of 1 
with its correspondingly larger sample size 
for a designated AQL (where available), may 
be used when designated or approved by the 
responsible authority. 


9.5 TYPES OF SAMPLING PLANS. Three 
types of sampling plans: Single, Double and 
Multiple, are given in Tables II, ITI and IV, 
respectively. When several types of plans are 
available for a given AQL and code letter, 
any one may be used. A decision as to type 
of plan, either single, double, or multiple, 
when available for a given AQL and code 
letter, will usually be based upon the com- 
parison between the administrative difficulty 
and the average sample sizes of the available 
plans. The average sample size of multiple 
plans is less than for double (except in the 
case corresponding to single acceptance num- 
ber 1) and both of these are always less than 
a single sample size. Usually thé administra- 
tive difficulty for single sampling and the 
cost per unit of the sample are less than-for 
double or multiple. 





10. DETERMINATION OF ACCEPTABILITY 


10.1 PERCENT DEFECTIVE INSPECTION. 
To determine acceptability of a lot or batch 
under percent defective inspection, the ap- 
plicable sampling plan shall be used in 
accordance with 10.1.1, 10.1.2, 10.1.3, 10.1.4, 
and 10.1.5. 


10.1.3 SINGLE SAMPLING PLAN. The 
number of sample units inspected shall be 
equal to the sample size given by the plan. 
If the number of defectives found in the 
sample is equal to or less than the acceptance 
number, the lot or batch shall be considered 
acceptable. If the number of defectives is 
equal to or greater than the rejection num- 
ber, the lot or batch shall be rejected. 


10.1.2 DOUBLE SAMPLING PLAN. The 
number of sample units inspected shall be 
equal to the first sample size given by the 
plan. If the number of defectives found in 
the first sample is equal to or less than the 
first acceptance number, the lot or batch 
shall be considered acceptable. If the num- 
ber of defectives found in the first sample is 
equal to or greater than the first rejection 
number, the lot or batch shall be rejected. 
If the number of defectives found in the first 
sample is between the first acceptance and 
rejection numbers, a second sample of the 
size given by the plan shall be inspected. The 


number of defectives found in the first and 
second samples shall be accumulated. If the 
cumulative number of defectives is equal to 
or less than the second acceptance number, 
the lot or batch shall be considered accept- 
able. If the cumulative number of defectives 
is equal to or greater than the second rejec- 
tion number, the lot or batch shall be rejected. 


10.1.3 MULTIPLE SAMPLE PLAN. Under 
multiple sampling, the procedure shall be 
similar to that specified in 10.1.2, except that 
the number of successive samples required 
to reach a decision may be more than two. 


10.1.4 SPECIAL PROCEDURE FOR RE. 
DUCED INSPECTION. Under reduced in- 
spection, the sampling procedure may termi- 
nate without either acceptance or rejection 
criteria having been met. In these circum- 
stances, the lot or batch will be considered 
acceptable, but normal inspection will be 
reinstated starting with the next lot or 
batch (see 8.3.4 (b)). 


10.2. DEFECTS PER HUNDRED UNITS IN. 
SPECTION. To determine the acceptability 
of a lot or batch under Defects per Hundred 
Units inspection, the procedure specified for 
Percent Defective inspection above shall be 
used, except that the word ‘defects’ shal! he 
substituted for “defectives.” 


11. SUPPLEMENTARY INFORMATION | 


1.1 OPERATING CHARACTERISTIC 
CURVES. The operating characteristic curves 
for normal inspection, shown in Table X 
(pages 30-62), indicate the percentage of 
lots or batches which may be expected to be 
accepted under the various sampling plans 
for a given process quality. The curves shown 
are for single sampling; curves for double 


and multiple sampling are matched as closely 
as practicable. The O.C. curves shown for 
AQLs greater than 10.0 are based on the 
Poisson distribution and are applicable for 
defects per hundred units inspection; those 
for AQLs of 10.0 or less and sample sizes of 
80 or less are based on the binomial distri- 
bution and are applicable for percent defec- 
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Ti. SUPPLEMENTARY INFORMATION (Continued) 


tive inspection; those for AQLs of 10.0 or 
less and sample sizes larger then 80 are based 
on the Poisson distribution and are applica- 
ble either for defects per hundred units in- 
spection, or for percent defective inspection 
(the Poisson distribution being an adequate 
approximation to the binomial distribution 
under these conditions). Tabulated values, 
corresponding to selected values of probabil- 
ities of acceptance (P,,in percent) are given 
for each of the curves shown, and, in addi- 
tion, for tightened inspection, and for defects 
per hundred units for AQLs of 10.0 or iess 
and sample sizes of 80 or less. 


11.2 PROCESS AVERAGE. The process 
average is the average percent defective or 
average number of defects per hundred units 
(whichever is applicable) of product sub- 
mitted by the supplier for original inspec- 
tion. Original inspection is the first inspec- 
tion of a particular quantity of product as 
distinguished from the inspection of product 
which has been resubmitted after prior 
rejection. 


11.3. AVERAGE OUTGOING QUALITY 
(AQQ). The AOQ is the average quality of 
outgoing product including all accepted lots 
or batches, pius all rejected lots or batches 
after the rejected lots or batches have been 
effectively 100 percent inspected and all de- 
fectives replaced by nondefectives. 


11.4 AVERAGE OUTGOING QUALITY 
LIMIT (AOQL). The AOQL is the maximum 
of the AOQs for all possible incoming quali- 
ties for a given acceptance sampling plan. 
AOQL values are given in Table V-A for 
each of the single sampling plans for normal 
inspection and in Table V-~B for each of the 
smgle sampling plans for tightened inspec- 
tion. 
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11.5 AVERAGE SAMPLE SIZE CURVES. 
Average sample size curves for double and 
multiple sampling are in Table IX. These 
show the average sample sizes which may be 
expected to occur under the various sampling 
plans for a given process quality. The curves 
assume r:G curtailment of inspection and are 
approximate to the extent that they are 
based upon the Poisson distribution, and that 
the sample sizes for double and multiple 
sampling are assumed to be 0.631n and 0.25n 
respectively, where n is the equivalent single 
sample size. 


11.6 LIMITING QUALITY PROTECTION. 
The sampling plans and associated proce- 
dures given in this publication were designed 
for use where the units of product are pro- 
duced in a continuing series of lots or batches 
over a period of time. However, if the lot 
or batch is of an isolated nature, it is desira- 
ble to limit the selection of sampling plans 
to those, associated with a designated AQL 
value, that provide not less than a specified 
limiting quality protection. Sampling plans 
for this purpose can be selected by choosing 
a Limiting Quality (LQ) and a consumer's 
risk to be associated with it. Tables VI and 
VII give values of LQ for the commonly used 
consumer’s risks of 10 percent and 5 percent 
respectively. If a different value of con- 
sumer’s risk is required, the O.C. curves and 
their tabulated values may be used. The 
concept of LQ may also be useful in specify- 
ing the AQL and Inspection Levels for a 
series of lots or batches, thus ixing minimum 
sample size where there is some reason for 
avoiding (with more than a given consumer's 
risk) more than a limiting proportion of de- 
fectives (or defects) in any single lot or 


batch. 
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TABLE X-H—Tables for sample size code letter: H 


LE SAMPLING PLANS 
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- OPERATING CHARACTERISTIC CURVES FOR SIN 
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- TABULATED VALUES FOR OPERATING CHARACTERISTIC CURVES FOR SINGLE SAMPLING PLANS 


TABLE X-tl 


Acceptable Quality Levels (nonnal inspection) 
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10.4.1 Selecting the Sampling Plan. Two factors are generally con- 
sidered in the selection of a Sampling plan: (1) the consumer and/ 
or supplier risk factor and (2) the economic factor. The miske pat 
rern of each sampling plan is represented by the OC curve for the 
plan. The OC curve for each plan is different, a property which 
provides an effective means for ascertaining the eptrectsor changes 
‘in sample size and acceptance number on the acceptance or rejection 
of a lot. The proper (with respect Co risk) sampling plan can be 
determined from studying the OC curve for each plan under consider- 
ation. By studying the OC curves, it is possible to compare the 
relative risks of two or more sampling plans for a given sampling 
situation. By virtue of the OC curve, sampling tables can be con- 
structed in which risks of incorrect decisions have been determined 
in advance, making it possible to select plans which will have risk 
factors that are acceptable to both the supplier and the consumer. 
The OC curve, then, can be used for classifying sampling plans from 
the standpoint of the protection afforded to the supplier (AQL plans), 
Penscumer (LO plans), or both. The economic factor must be considered 
each time a sampling plan is to be selected and, of course, becomes 
more and more important as the cost of testing goes up. This factor 
becomes especially important when, because of the high cost of test- 
ing, sample size must be limited to a degree which forces a compromise 
of the risk requirements specified for the sampling plan. Another 
approach to selecting sampling plans is used by some organizations 
which handle many types of items. Instead of selecting a sampling 
plan on an item by item basis as the above procedure suggests, a 
standard operating procedure is established whereby a particular 
very stringent sampling plan (probably acceptance number of zero 

and large sample size, pechapse. Che genet ce population) is designated 
to use when inspecting any quality characteristic that may be a 
critical defect, a second but less stringent sampling plan is desig- 
nated to use when inspecting any quality characteristic or group of 
quality characteristics that will be at worst a major defect(s), and 
a third and still less stringent sampling plan is designated to use 
when inspecting any quality characteristic or group of quality Sia 
acteristics that will be no worse than a minor defect(s). 


Hor4.2 Effects of Changes £9 the Sampling Plan on the OC Curve. 

A sampling plan and its associated risks are completely defined by 

the lot size, sample size, and acceptance number. The lot size, 
except in the case of very small lots, has relatively little import — 
ance in most cases in determining the risks associated with any 

given sampling plan. Thus, sample sizes and acceptance numbers are 
the two important factors which influence the risk pattern of sampling 
plans. If the risks of a tentative sampling plan are considered un- 
satisfactory, the question which follows is: "What changes must be 
made to obtain the desired sampling protection?" This can be answered 
by considering the effect on the OC curve of changes in the sampling 
plan. To understand the effect of such changes, a more detailed study 
of the OC curve (see Figure 2) is appropriate. From examination of 
this curve it is seen that if lots to be inspected are 2% defective, 
approximately 90% of the lots are expected to be accepted, whereas if 
the lots submitted are 8% defective, about 10% of the lots are 
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expected to be accepted. If 2% defective and 8% defective represent 
good and bad quality lots, respectively, the good lots will be re- 
jected 102 (100 - 90 = 10) of the time (producer's risk) and bad 

lots accepted but 10% of rhe time (consumer's risk). This rejection/ 
acceptance frequency will occur by chance. [If this frequency is 
intolerable, appropriate changes to the sampling plan are required. 


70.4.3 Changes in Sample Size. An increase in sample size results 
in a steepening of the OG curve, as indicated in Figure 3. The 
steeper the OC curve, the greater the power of the sampling plan to 
discriminate between "sood" and "bad" quality. Figure 3 clearly 
illustrates the effect that increasing sample size has on waking 
the OC curve “steeper... 


mona.4 @Ghanees in Acceptance Number. Figure 4 illustrates the 
effect of changes in the acceptance/rejection numbers on the OC 
curve. Jn general, the effect of increasing the acceptance Mees 
is to shift the location of the entire OC curve to the right. 
Changing the sampling plan in this way generally increases the 
probability of accepting a lot at a given quality level. 
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10.4.5 Simultaneous Change of Sample Size and Acceptance Number. 


If it is desired to have more accurate disposition of the lots whose 
percent defective is close to the selected quality level (the AQL 

or the LQ for example), the sample size must be increased to provide 
more discrimination. Also, the acceptance number must be selected 
which will yield the OC curve that is properly located ebout the 
"desired" quality level. Thus, if the degree of discrimination of 

a given plan is considered adequate, but the probability of accepting 
a lot at a given quality level is too great (i.e., the plan is “too 
loose ') Or Coo small (i.e., tne plan is “too tight"), proper adjust- 
ment is made by selecting the appropriate acceptance number. Usually 
in practice, if a sampling plan is desired which nas certain desirable 
risk characteristics, both sample size and acceptance numbers must be 
simultaneously adjusted (See Figure 5). In order to make proper 
adjustment, however, the effect of each must be understood. 
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SECTION 20: SEQUENCE OF OPERATIONS IN USING MIL-STD-105 


A typical sequence of operations in using the sampling pro- 
cedures and tables for inspection by attributes of MIL-STD-105 
poeleebustrated by Table C which follows. This table assumes a 
mequirement for single sampling. 


Papi e, Cc : Secuence of Operational steps 


Steps 


3 ee Deternine lot size. 


2. Determine inspection 
ewe le. 


3. Determine sample size 
code letter. 


4, Determine sampling plans. 
pewmeestcablish severity of 
wspecti10n. 


6. Determine sanple size 
and acceptance nunber. 


i “oelece sample. 


Gaeelnspecte sample, 


Saenelcond inspection 
results. 


t- 


-~ 


Explanation 


P SGGC size -controlled= by lot 

fOumMoItioOn eriteria contdined in 
PLCcUnRcnenENdocuments, Otherwise, 
establish by agreement between 
responsible authority and supplier. 

Ze If the Lten specification does 

not give the inspection level, use 
inspection level II. 

3. Found in Table I, MIL-STD-105, 

DaSeCd yon hotest se aid inspection 

evel 

4. Single sampling generally selected. 
Double or multiple sampling may be used. 
5. Normal inspection generaliy used at 
StQre OCheconcracwmonr Product ioa. 

6. Assuming normal inspectiun and given 
the specified AQL value and the sample 
size code letter, the sample size and 
acceptance number are found in Table 
PISA, MIL—-STD— 105. 

7. The sample, consisting of the nunber 
of units of product as determined from 
Table II-A, MIL-STD-105, is selected at 
Fandom teometnecelLoOtas Additionally,» any 
obvious defectives that have not been 
selected for the inspection sample are 
removed from the lot (but are not in- 
cluded ins tnescanple)eueuoee para, 14.2) 
8. The defectives (or defects) are 
counted, Lf this count does not excecd 
thes acceptance number (ac), the entire 
IGtmicedcceprec. wet tne Count equals 

Or exceeds tie rejection number, the 

bot. tsorejyected. 

Pe Gomputeceseimated 9rocess average 

bi wreguired=byeoperacing procedures. 
Maintain ececoramor accepe/reject deci- 
SlOnS In opder Ehat SWitching rules 

may be followed. 
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Steps Explanation 


10. Resubmit lot. DO, Li tne Lot is not accepted, it 
may be resubmitted for acceptance 
inspection only after all units of 
the lot are reinspected and all 
defective units removed or reworked. 


Example 6: Obtaining a Plan. Suppose the AQL is 1.0, the inspection 
level is II and the lot size is 2,500. The first thing required is the 
sample size code letter (usually called simply the code letter, for 
BHlOmt.)) For a l6t size of 2,500 and inspection level II, Table I 

gives the code letter as K. In the appropriate master table (Table 
III-A), it is found that the sample size for single sampling is 125. 
AQLs for normal inspection are given along the top of the table, and 
under the value 1.0 we find the numbers 3 and 4 given under the heading 
Ac Re (which stand for acceptance wuumvexr and rejecticn number, respect- 
ively). The sampling plan required is: 


Sample size iS 
Acceptance number 3 : 
Rejection number 4 


Alternatively, Table X-K-2 could be used. Again the sample size 
of 125 is found; and in the column for AQL 1.0 are found the accep- 


/ 


tance and rejection numbers 3 and 4 as before. 


Example 7: Arrows in Tables If, Lif, and [V. Suppose the AQL is 0.40, 
the inspection level is I, and the lot size is 230. Table I gives the 
code letter as E. Using Table LII-A, it is found that there is no plan 
for letter E and AQL 0.40 but a downward pointing arrow that directs 

us to letter G instead, and the required plan is: 


Sample size 32 
Acceptance number 0 
Rejection number 1 


Alternatively, the specifying of code letter E leads us, in the ex- 
tended tables, to Table X-E-2. But this page has no column for 

AQL 0.40. Instead, the symbol of and inverted triangle appears 

for AQLs less than 1.0. This triangle refers to the footnote "Use 
next subsequent sample size code letter for which acceptance and re- 
jection numbers are available." If the triangle is thought of as an 
arrowhead, it is pointing towards the edge of the page to be turned. 
This leads to letter F where again AQL 0.40 is not given, and on to 
letter G to rind the same plan as before. It is very important to 
remember that if a triangle or series of triangles directs you from 
One page to another of the extended tables, or an arrow directs you 
from One row to another of the master tables, the sample size to be 
used is the one given for the new page or the new row arrived at and 
not the one given for the original page or row [9.4]. Where upward 
pointing arrows or triangles are found the meaning is similar. The 
triangles again point to the edge of the page to be turned. 
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EXCERPTS FROM OFPP PAMPHLET NO. 4 


Twe chapters frem OFPP Pampiet No. G "A Guide for 
Writing and Administering Performance Statements of Work for 
Service Ccn*tracts", are provided in this appenaix. These ar: 
Chapter 4, The Surveillance Plan, aude Chea peca §55,. Deing 
Surveillance. Chapter 4 addresses quality assurance puan 
feemanmeard iilustsates che role that MIL-STD-105D plays in 
the sampling process. Chapter 5 elaporates on ths adminis- 
eBajive Erocedures necessary if conducting inspection. It 
should be noted that the OFPP approach is equivalent tc Air 
Mmoemee incpection procedures given in AFR 400-26. Finally, 2 


Performance Requirement Summary (PRS) form is shown. 
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CHAPTER 4 


THE SURVEILLANCE PLAN 


4-1. Basic Approach. This chapter describes the major 
contents of a surveillance plan. There are three key ideas 
that are the basis for a surveillance plan. 


a. Management By Exception. Quality assurance relates 
to the output service provided by the contractor. As pointed 
out earlier, the output service can result either from a 
contractor-developed procedure or from an government specified 
procedure. When the procedure is specified by the government, 
compliance with the procedure is the desired output service. 


(1) When the output is based on a contractor 
developed procedure, the procedures are only looked at on a 
by-exception basis; that is, satisfactory performance of the 
Output service as specified in the contract normally 
Indicates that the contractor is using satisfactory 
procedures. The government should be concerned only when 
services are not adequately performed. 


(2) In this case, the inspector looks beyond the 
level of services provided only to determine if the problem 
1s caused by the government or the contractor. If government 
provided items to the contractor's operation (Such as, 
parts, equipment, or facilities) are at fault, action must be 
taken through government channels to correct the problem. No 
action will be required of the contractor. When the problem 
ls the contractor's fault, the contractor 1s told to take 
corrective action. 


b. Performance Indicator. The level of contractor 
provided services is monitored by checking the performance 
values in the statement of work (SOW). As described in 
chapter 2, a performance value is a feature of the service 
that can be measured by a number. For example, two 
important pertormance values in vehicle maintenance and 
vehicle operations are vehicle out-of-commission (VOC) rate 
and taxl response time. 


c. Problem Lecation. When performance values show 
that the service 1S not adequately performed, the QAE uses 
decision tables to locate the problem. The tables provide a 
logical sequence to find the problem cause. Basically, they 
are a set of pointers which should find the problem's source 
in a step-by-step fashion. The construction and use of 
decision tables are described in paragraph 4-4b. 
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4-2. Surveillance Information Sources. There are four 
principal sources of information for surveillance: management 
information systems, random sampling, checklists, and formal 
customer complaints. The following sections describe the 
information sources in detail. 


a. Management Information Systems. In a few instances, 
an existing management information system (MIS) may be 
avallable as a means of surveillance. When a MIS is 
available, as in the case of the Air Force's vehicle integrated 
management system (VIMS) in the vehicle maintenance area, it 
can collect information on performance values which can 
be used instead of random sampling data. | 


(1) Management information systems usually collect 
information for 100 percent of the activities for a 
specified period of time. This information can be compared 
to a contract standard. On the basis of this comparison, 
performance can be judged and the performance for the 
specified period accepted or rejected. 


(2) For example, the vehicle out-of-commission (VOC) 
rate is computed every month by the VIMS. A simple 
comparison of the VOC rate with the maximum acceptable VOC 
in the SOW explains 2 great deal about the level of 
maintenance service supporting the base vehicles and 
organizations. 


(3) By way of caution, however, one must check the 
data input into a MIS if the system 1s maintained by the 
contractor. If one 1s going to use a MIS to check the 
contractor, make sure the MIS contains reliable data. 


b. Random Sampling. The most frequently used way of 
service contract surveillance 1s random sampling. Services 
are sampled by the QAE to determine 1f£ the contractor's 
level of performance is acceptable. Acceptance sampling is 
done, basically, to determine a course of action: that is, 
whether to accept or reject the contractor's level of 
vecformance during a given period of time. If it rejects 
performance, certain actions are started. If it accepts 
performance, no action ls taken. 


(1) The basis for doing random sampling is 
MIL-STD-105D, Sampling Procedures and Tables for Inspection 
by Attributes which 1s widely understood and used by both the 
government and contractors. It 1s based on the concept of an 
attribute. An attribute is a feature of a service which 
either does, or does not, match a standard (for example, a 
taxi is on time or it is not on time). 





(2) When sampling by attributes, a certain number 
of observations will match the standards and the remaining 
number will not match. Therefore, attribute sampling is 
useful for describing how a job is done, in terms of defects 
per hundred observations, or percent defective. Using this 
concept, sampling for a performance indicator can be 
developed by proceeding through a number of formal steps 
based on MIL-STD-105D. The use of these concepts is 
described in paragraph 4-3, Sampling Plan. 


¢c. Surveillance Checklists. Checklists are also used 
to check contract performance. They must be used sparingly, 
however. The use of the MIS and random sampling are preferred 
information sources. Checklists help in surveillance of 
contract requirements that happen infrequently. (For example, 
1f a contractor is required to perform a service once a 
month, this service would be included on a checklist.) Any 
service that 1s not provided on a daily basis should be 
considered for inclusion on a checklist unless a MIS can be 
used to determine the quality of the service. 


d. Formal Customer Complaints. Even the best 
surveillance plan will not allow the QAE to check all aspects 
of the contractor's performance. 


(1) Formal customer complaints are a means of 
documenting certain kinds of service problems. The way to 
get and document customer complaints needs to be carefully 
planned by the persons checking the service contract. 


(2) Customer complaints are not truly random. They 
are seldom used to reject a service or deduct money from the 
GOnetractor. 


(3) When random sampling is the chosen method of 
Surveillance, a customer complaint cannot be used to satisfy a 
random observation. However, it can be used as further 
evidence of unsatisfactory performance if random sampling 
shows that the specific service 1S unsatisfactory. These 
complaints can be used to decide if action other than a 
deduction should be taken. 


(a) Getting Customer Complaints. An 
aggressive customer complaint program, once established, 
needs to be briefed to every organization that receives the 
contractor's services. An Operating instruction should be 
given to each organization outlining the customer complaint 
program, the format and the content of a formal customer 
complaint, and the action which can be expected from those 
assigned to watching and managing the service contract. 


oo 





(6) Documenting the Customer Complaint. 
Normally, each customer complaint is brought, either in 
person or by telephone, to the person checking contract 
performance. Enter information about the complaint into a 
Customer Complaint Record, similar to the sample shown in 
figure 4-1. The record contains the following information: 


l Date and time of complaint. 


2 Source of complaint - organization and 
individual. 


3 Nature of complaint (narrative 
description). 

4 Contract reference of complaint related 
services. 

5 Valid complaint (Yes or No). 

6 Date contractor informed of complaint. 

7 Action taken by contractor. 

8 Signature of the person receiving and 


validating the complaint. 


4-3. Sampling Plan. As a rule, a plan contains information 
on the acceptable quality level, lot size, sample size, and 
rejection level. It states the number of units from each lot 
to be inspected (that is, the sample size). It also states 
the criteria for determining the acceptability of the lot 
(acceptance and rejection numbers). This information is used 
to build the sampling guide which are the major products in a 
survelllance plan for a service contract. 


a. Beginning the Plan. To begin building a sampling 


plan, go to the Performance Requirements Summary developed 
Guring the "Write Statement of Work" step, chapter 3, fiaure 
3-1. 


(1) This chart contains the required services, the 
standards, and acceptable quality levels. At this time 
decide how the services will be checked (what information 
source or method of surveillance will be used). 


(2) Show these decisions on the chart. For each 


service where random sampling is used, complete the steps 
described below. 
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CUSTOMER COMPLAINT RECORD 








Date and Time of Complaint: 21 Jan 1979 / :1005 










Source of Complaint 





Organization: 382 Bomb Wing/LGC 












Individual: 





Capt John Murry 






Nature of Complaint: Calledwrecker and it did not arrive until 
3 hours after the request. 






Contract Reference: F-5, para 5.1.1.2.5 and Performance 
Requirements Summary. 











Yalidation: Contract requires a 7] hour response time. Complaint 


is valid. 










Date and Time Contractor Informed of Complaint: 27 Jan 79/:1030 






Action Taken by Contractor: 
Contractor had a person out sick and did not have a back up driver. 


He has now developed a roster of back up drivers who can operate a 
wrecker, 


Received and Validated By: H. Smyth/QAE 


Figure 4-1. Customer Complaint Record. 
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b. Deciding on the Acceptable Quality Level (AQL). The 
AQL 1s the highest number of defects per hundred, highest 
percent defective or highest number of defects that can be 
allowed for any service performance indicator. There are 
Only a limited number of AQLs listed in MIL-STD-105D but, in 
virtually all cases, one will be close enough to control the 
contractor's level of service. 


(1) The first step in designing a sampling plan 
under MIL-STD-105D is the selection of a realistic AQL. No 
service can be perfectly performed. The AOQLS placed on the 
Performance Requirements Summary in figure 3-l, must be 
adjusted at this time. 


(2) Find the closest AQL from figure 4-2 and use it 
to replace the original AQL on the Performance Requirement 
Summary. For example, the AQL for taxi service might have 
been 5 percent. This would be changed to 4 percent or 6.5 
percent since 5 percent does not appear in the figure. 


c. Determining the Lot Size. To determine the sample 
size, the lot size must be known. The lot is how often the 
contractor provides the service in a period of time. 


(1) To determine the lot size, estimate (or count) 
the frequency of the service to be sampled, during the period 
it 1s to be sampled. Thus, if scheduled bus service 
timeliness is the service being sampled, and a sample is 
taken each month, the lot size is the number of times that 
are available during the month to observe bus timeliness. 

In this case, it would be the number of times the buses go 
around all the routes each day, multiplied by the number of 
days in each month on which the bus routes operate. 


(2) In the case of workorders, the monthly lot size 
Can be estimated from historical information on file. The 
projected workload data gathered in chapter 2 is used to help 
determine lot sizes. 


d. Determining the Sample Size. Use figure 4-3 to 
identify an appropriate sample size for a given lot size. 


(1) Use the normal sample size column unless there 
1s a limited number of QAEs or unless the cost of an 
inspection suggests the use of the medium or small sample 
size column. 


(2) Use the medium or small sample size, if 


inspections for a particular service are lengthy or hinder 
the contractor's ability to provide service to customers. 
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i eS a ga ee rr er 


Allowable Acceptable Quality Levels 





0.010 % ; 71.0% 
0.015 % 1.5 % 
0.025 % 259% 
0.040 % 4.0 % 
0.065 % 6.5.% 
0.10 % 10. 4% 
0.15 % 15. 4% 
Osco ts 25. % 
0.40 % 40. % 
0.65 % 65. % 


Figure 4-2. List of MIL-STD-1050 Acceptable Quality Levels. 
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tot Size Normal Sample Medium Sample Small Sample 





size S$1ze Size 

2 ae 2 2 2 
9-15 3 2 2 
16-25 5 3 3 
26-50 8 5 5 
51-90 13 5 5 
91-150 20 8 8 
151-289 32 13 13 
281-500 50 20 13 
501-1,200 80 32 20 
1,201-3,200 125 50 32 
3,201-10,000 200 80 32 
10,001-35 ,000 315 125 30 
35,001-150,000 500 200 80 
150,001-500 ,000 800 315 80 
500,000 and over 250 500 125 


Figure 4-3. Sample Size. 





e. Selecting the Rejection Level. Use MIL-STD-105D 
to identify the acceptance and rejection level for the 
sample size (see figure 4-4). To use the figure, begin with 
the known values for the AQL and the sample size. 


(1) Find the selected sample size (in the sample 
slze column) and read across that line to the column for the 
selected AQL. At that point there will either be two numbers 
Or an arrow pointing up or down. 


(2) If there is an arrow, follow the direction of 
the arrow until it leads to a pair of numbers. Of the two 
numbers at the intersection or at the end of the arrow, the 
number on the left (Ac or accept) indicates the maximum 
number of defects which can occur in a_ sample and still 
permit tne total group or lot to be judged acceptable. 


(3) When there is no accept of reject number for a 
given sample size and AQL, following the arrow will also 
cause a change in sample size. For example, with an AQL of 
1.5 and a sample size of 20, the sample size would become 32. 


(4) The number on the right (Re or reject) 
indicates the minimum number of defects that occur ina 
sample which causes the total group or lot to be judged 
unacceptable. For example, suppose the sample size is 
determined to be 32 and the AQL has been set at 6.5 defects 
per hundred. Find the number 32 in the sample size column 
and read across that line until the AOL column for 6.5 has 
been reached. The two numbers at that intersection are 5 and 
5. 


(a) In other words, the number on the left, 5, 
is the number of defects which can be found in a sample and 
Still permit acceptance of the lot. 


(b) The number 6, to the right of 5S, is the 
smallest number of defects needed to declare the lot 
unacceptable and subject to further check, using the 
decision tables. 


4-4. Developing the Sampling Method. The final thing to be 
decided in sampling is how the sample will be drawn. The 
objective in the method is to insure that the sample is 
random (that is, that all services have an equal chance of 
being selected). To achieve random selection, use a random 
number table, as explained in the following examples (see 
attachment 1 for the whole table). Most items will fall 
into one of these examples. 
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(3) Go to the next number, 60756. The last part of 
this number, 756, falls within the brackets one is looking 
for, so workorder 756 is selected to be sampled. The next 
random number group is 92144. Since 144 is not within the 
brackets, move to the next group 49442. Again, 442 is not 
within the brackets and therefore is not selected to be 
sampled. 


(4) This process would be continued until three 
workorders are selected. 


c. How To Use The Random Number Table To Identify 
Random Sample From a List. If a number of items need to be 
sampled that are not consecutively numbered, the simplest 
solution is to list the identifiers, for all the items in the 
lot, in a column, on a piece of lined paper. 


(1) Next, number the lines consecutively, beginning 
with the number one hundred. Now use the random number table 
to draw the sample from the line numbers. A selected line 
number leads to the identifier located on that line and that 
identifier tells which item to sample. For example, if one 
chooses to sample a set of workorders with attached sales 
slips, one is not going to have to have a set of 
consecutively numbered workorders because not every workorder 
has a sales slip attached. 


(2) List the workorders with sales slips ina 
column, number each line in the column, and randomly select 
enough line numbers to make up the sample. 


d. How To Use The Random Number Table To Identify a 
Random Sample of Days. Suppose one wants to identify 4 days 
in the month on which to sample something. The days of the 
month can be numbered 01 to 31 (or less, as appropriate). 
Begin in the random number table in figure 4-5 at 77452. 


(1) It is best to wse a starting point different 
from the one used in the previous example but for the purpose 
of this example it is being used again. 


(2) One can move down the column from random number 
group to random number until the first number between 01 and 
31 is spotted. In this case, it is 23216 or, using the rule 
to discard the numbers to the left of the number of digits, 
Simply 16. Thus the 16th day of the month is selected for 
sampling. 


(3) Continuing in this fashion, one discovers that 
Sev sl, Or Samoly Ji, Or the 3lst, 1s the next day for 
Sampling. Proceed in this manner until’ the four days for 
Sampling have been identified. 
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(4) If it is not desirable to sample on weekends, 
discard those days selected that happen to fall on a weekend 
and continue that selection until the proper number of days 
has been selected. 


e. How To Use the Random Number Table To Identify a 
Random Sample of Times of Day. If one wants to select random 
times of day to sample a service such as taxi or bus service, 
use the 24 hour clock. 


(1) If there are any constraints during each 
24-hour period, take them into consideration. For example, 
Suppose that base bus service operates bDetween 0700 and 2345. 
In this case, go through the number table until one finds a 
group of four numbers that correspond to an acceptable time 
between 0700 and 2345. Again, uSing figure 4-5, and 
proceeding across the line from the initial number, one comes 
to 60756 or 0756 hrs as the first random time. 


(2) The next random number is 92144 or 2144 hrs. 
The number 1S good and so one schedules an observation for 
2144 hrs. 


(3) Proceed in this manner until the desired number 
of sample times have been identified. 


f. How To Insure Variety in the Use of the Random 
Number Table. The use of variety in the random number table 
ensures that detectable patterns do not occur. 


(1) Besides starting at different random points and 
alternating the patterns for finding a string of random 
numbers, the user may, at some point in time, wish to use the 
first significant digits instead of the last. 


(2) For instance, in the random number group 77452 
one has customarily used the last three digits (that is, 452) 
when looking for a random number with three digits. But 
there is no reason why one could not for a period of time use 
the first three digits, or 774. 


(3) Success in using the tables requires 
consistency but also variety. The above information should 
ensure that the tables are properly used and that the sample 
1s randomly drawn. 


4-5. Surveillance Plan Products. Several written documents 
are included in a surveillance plan: 


a. Sampling Guides. A sampling guide is used for 
Surveillance. It is used in a Surveillance plan to present 





a. Use Of The Random Number Table. The random numbers 
in figure 4-5 are arranged in groups of five numbers (51259, 
974252, and so on). 


(1) To use the table, begin by picking at random a 
group of numbers on any page of the table. This is usually 
done by closing the eyes and pointing with a pencil or 
finger to some initial group. 


(2) To identify additional random numbers, follow 
a pattern. Go along a given line to its end and then along 
the next line to its end and so on through the table until 
enough numbers have been selected or until the table ends. 


(3) If the table ends and there are still more 
numbers to select, go back to the beginning of the table and 
continue uSing the same pattern. Use various patterns 
alternately; for example, use lines for one sample, use 
columns for the next sample, and use a diagonal pattern for 
the third sample. 


b. How To Use the Random Number Table To Identify a 
Random Sample of Consecutively Numbered Workorders. Suppose 
one has to identify a random sample of 3 workorders for 
inspection. This can be done at the beginning of the month 
(before the workorders are written) or at the end of the 
month (to select workorders already on file). 


(1) If there are, or might be, 200 workorders to 
select from, then one begins by listing the lowest workorder 
number (Known or projected). This could be #001 or possibly 
#743, for example. 


(a) List the highest workorder number (known 
or projected); in this case, it could be #200 or #943. With 
these boundaries now enter the random number table to the 
first group of numbers. For this example, use workorders 
numbered #743 to #943. 


(b) If the last three digits in the first 
group of random numbers is not between 743 and 943, discard 
that group of numbers and go to the next group. 


(2) Again, using figure 4-5, if one starts at the 
initial 77452, disregard the two numbers to the left of the 
three significant digits, or in this case, 77. The 
remaining number is 452. Since this 1S not between 743 and 
943, go to the next group in the same line which is 16308, 
again, discard the leftmost two numbers, and the number is 
308. This iS again too low. 
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Figure 4-5. 





the information needed to sample the performance of a 
Particular service. Information for the sampling guide is 
developed while the sampling information is being derived for 
the sampling plan. The steps involved in developing 

Sampling information are described in paragraph 4-3. As 
shown in figure 4-6, a sampling guide has these sections: 


(1) A statement of the AQL and its meaning in 
layman's terms. 


(2) The lot size for sampling. 
(3) The sample size. 


(4) A description of the sampling procedure 
which tells how the service will be sampled. 


(5) An explanation of the inspection procedure 
which tells what will be checked during the inspection of the 
sample. 


(6) Acceptable performance criteria which 
States the acceptance and rejection levels. 


b. QAE Decision Tables. Once a problem has been 
discovered, the inspector must turn to a decision table and 
use the information in that table to aid him in finding the 
source of the problem. The decision table lists the symptoms 
of the problem and identifies the possible sources of the 
problem. Questions are established for each potential source 
to determine the contributing factors. A decision logic 
entry is worked up for each required service. As soon as it 
ls considered satisfactory, the information is transferred to 
the decision table. An example of a decision logic entry is 
shown as part of a decision table in the sample in figure 
4-7. 


c. Checklists. There are two main uses for checklists, 


(1) Tally Checklists. Tally checklists are used to 
document all sample observations: made during a sampling 
period. Checklists may be preprinted with any format which 
contains the following information: 


(a) Contract requirements - a statement of the 
service being inspected. 


(b) Date, time, entry for each observation. 
(c) Observation identifier of applicable 


workorder number, bus stop, or sales slip number, meal 
period, etc. 





VO Sampling Guide #3 
Vehicle Condition Monitoring 


1. Acceptable Quality Level (AQL): 10% 

n the long run there must be no more than 10 defects per. hundred 
vehicles. 
Zz. BeOt Size: vehicles operated by the contractor. 


3. Samole Size: vehicles operated by the contractor. 








4. Samoling Procedure: 

At the beginning of the month, list the registration numbers of al] 
contractor operated vehicles on a sheet of ruled paper. Beginning with 
the number 100, number the lines on the oaper to correspond with the 
vehicle registration numbers. Using the random number table select line 
numbers equal to the sample size. The vehicle registration numbers on 
these lines indicate the vehicles to be sampled during the month. Schedule 
the inspections evenly over the month. 


5. Inspection Procedure: 

Insvect the vehicles using vehicle/equipment discrepancy and 
maintenance report as a quide (see AFM 77-310, Vol II, Chap 6). Record 
defects per vehicle for each of the inspected vehicles. Any defects 
found not already noted by the contractor shall cause the observation 
to be recorded as unsatisfactory. 


6. Performance Criteria: 


a. Performance is acceptable when or less defective vehicles are 
discovered per month. 


b. Performance is unacceptable when or more defective vehicles 
are discovered during a month.. 


7. Phase-In Period: During the first two months of the contract the 
following AQL’s (paragraph 1) and performance criteria (paragraph 6) 


mmm Tove 


a. AQL: 15% 


b. Performance is acceptable if or fewer defects are discovered 
per month. 


c. Performance is unacceptable if or more defects are 
discovered per month. 


Figure 4-6. Sampling Guide. 
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(d) Result of observation - either 
satisfactory or defective. 


(e) Any pertinent comment for an observation. 
An example of a checklist for sampling is shown in figure 
4-8. 


(2) Surveillance Activity Checklists. When a 
specific service or procedure occurs rarely or is not 
important enough to survey on a continuing basis, use a 
Surveillance Activity Checklist. This type of checklist must 
be prepared and included in the surveillance plan. An 
example of this kind of checklist is shown in figure 4-9, 


4-6. Contract Administrator's Plan. The contract 
administrator has an obligation to see how well the QAE is 
doing the job. The contract administrator must also make 
some independent checks of contractor performance, preferably 
by using the same techniques that go into the design of the 
QA surveillance plan. (This plan 1s completed at the same 
time as the surveillance plan.) As a minimum this plan must 
call for a quarterly review of the QAE's use of sampling 
guides and an annual review of surveillance activity 
checklist items. 
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Vehicle Operations Sampling Guide #3 


Vehicle Condition Monitoring 


Registration . 
Date | Time |Satisfactory Un- 
Number pate ‘a sfactory Remarks 


8 7305 dct| 1530 /_ | _—__ 
B 9763 Le a 
B 8754 ; Oct os a 
B 0010 Soctlosoo} x | | | 
B 8764 5 0 i i es ae ae 
B 7707 5 Oct} 1345 x en eee 
8 7706 lsoct}{igoo}] x | | 

5 eee 


18 Oct] 1000 












Sader! 1025 a. ae ee Door handle KkLG 
ens oc en ee 









8 Oct! 1600 | 


Figure 4-8. Sample Tally Checklist. 
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CHAPTER 5 


DOING SURVEILLANCE 


S-l. Surveillance Methods. This chapter tells how to do 
Surveillance once the plan is written. It tells how to build 
a monthly schedule, how to use the surveillance plan, and 
what to do when there is poor contractor performance. This 
Chapter applies to Quality Assurance Evaluators (QAEs) and 
contract administrators. 


5-2. Building A Schedule. A Surveillance plan is organized 
to facilitate use by the QAE. The QAE is responsible for 
developing a monthly schedule for activities, based on the 
Surveillance plan's requirements. Complete the Quality 
Assurance Evaluator Schedule by the last workday of the 
preceding month and send a copy to the contract administrator 
and the functional area chief for their information and 
review. Each QAE builds a schedule by filling in the blocks 
on the schedule. Specific instructions for filling out the 
schedule are provided below. 


a. Quality Assurance Evaluator Schedule. An example of 
a surveillance schedule is show in figure 5-l. The left-hand 
side of the schedule divides the sheet into days of the week. 
This example shows only a 7-day schedule. The QAE must make 
up enough sheets to include each day of the month. Along the 
top of the schedule, insert the items to be checked during 
the month. Along the bottom of the schedule, indicate the 
number of observations to be made during the month (that is, 
how often a MIS is checked, how many samples will be taken, 
how often a surveillance checklist will be used). 


Dae Filling in And Updating the Sehedule. To fill in 
the inspection schedule, the QAE refers to the sampling guide 
for each service being monitored. The sampling guide is used 
with the random number table to determine the inspections 
(observations) to be made during the month (see chapter 4, 
paragraph 4-4). 


(1) Contract surveillance must cover all hours of 
Operation. Random observations are scheduled at night, on 
weekends and holidays when services are performed during 
these periods. Areas that are monitored on a set schedule 
(for example, VIMS standards and analysis reports) are 
included in the monthly schedule. This monthly schedule 
shows where and what the QAE is monitoring at all times. 
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(2) Post any changes to the schedule weekly and 
send copies to the contract administrator and to the 
functional area chief. Document and explain the reasons for 
each change. Actual surveillance activity recorded on the 
Surveillance checklist must be comparable to the monthly 
schedule. 


(3) As updated, one must be able to conduct a 
complete audit trail from the monthly schedule, to observing 
the QAE perform sampling, to completion of the surveillance 
checklist. 


(a) There must also be a correlation between 
contractor performance versus standards, AQLs, checklists and 
actions taken by the contract administrator. The sample in 
figure 5-1 shows the schedule for one week. The QAE 
completes the blank forms, indicating week of (Monday through 
Sunday), and enters the time, observation, and check (if 
pertaining to a checklist), in the blocks corresponding to 
the ltem and day. 


(b) After it 1s completed and filled in, this 
form is "FOR OFFICIAL USE ONLY" and must not be shown to the 
contractor. 


5-3. Doing Surveillance. Doing surveillance involves using 
the surveillance plan called for in the monthly schedule. 
Use the following procedure to record observations and take 
action when the contractor's defects exceed the allowable 
number. 


a. Recording Observations. Monthly tally and 
surveillance checklists are used for each sampling guide 
and less frequently checked services. They are used to tally 
information on scheduled observations and defects noted. 
Each observation in the sample is recorded on the checklists, 
and the documents then become a formal government record for 
later reference. 


(1) When random sampling guides are used, the tally 
of observations and defects at the end of each month are 
compared to the acceptable number of defects appearing in the 
Sampling guide. 


(2) The contractor is told each time an error is 
found during scheduled observations and asked to initial the 
observation recorded on the checklist. 


(3) Errors found in Services not scheduled for 
observation should be brought to the contractor's attention 
but not used to count as a defect for determining 1f£ the AQL 
has been met. 
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(4) Checks done with a surveillance activity 
checklist are likewise recorded. , 


b. Potential Unsatisfactory Performance. If the 
sampling guide or surveillance activity checklist indicates 
that the number of defects is too high, the QAE goes to the 
decision table for that service indicator. 


(1) The QAE must locate the specific service that 
is unsatisfactory. The table will identify the possible 
causes of the unsatisfactory performance and list a number of 
questions which, if answered, will probably pinpoint the 
source of the problem. 


(2) The decision table helps the QAE identify the 
problem so that, among other things, a meaningful evaluation 
can be made of the contractor's explanation and corrective 
action. For example, if the contract specifies a maximum 
out~-of-commission rate for vehicles of 8 percent, and the 
rate was 10 percent, examination may reveal the excess was 
caused by excessive vehicle down for parts (VDP). This could 
have been caused by the government's inability to provide 
timely parts support. 


(3) In such a situation, the contractor may not be 
at fault. If, on the other hand, the excessive VDP was 
created because the contractor ordered the parts on a routine 
Eromicy rather than priority, lt might be the contractor's 
fault. The decision tables will assist the QAE in making 
such a determination. 


c. Documenting Unsatisfactory Performance. I€£ 
performance in any area is judged unsatisfactory, the 
contractor is required to respond to a Contract Discrepancy 
Report (CDR). See sample in Figure 5-2. 


(1) The QAE prepares the form and sends it to the 
contracting officer, who signs and sends it to the 
GEneractor. 


(2) When completed and signed, the report, along 
with the tally checklist or surveillance activity checklist 
become the documentation supporting payment, nonpayment, or 
other necessary action. 


5-4. Taking Action. The QAE may check the contractor's 
performance and document any non-compliance, but only the 
contracting officer may take formal action against the 
contractor for unsatisfactory performance. 
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CONTRACT DISCREPANCY REPORT 


1, €OnNTRS2CT muweka 


F12345-79-98765 
m 01 (Canwetias and Mermmgere Nowe) » #ACmr (Nome af QA) 


Vehicles Ine. - Mr. Travis Me. Smyth 


29 agraatg QRs2e MOTIFIC ATION @EKTwan dO 8" G2AHTA @ecTrOnm CCmeAL arn 

"Vsest 79 | vseoe7s | | s seat 79 [to sone 79 _| 

P. DISCREPANCY OR PROBLEM (Desertaee wt Getasl: imaivme cvlerwice 1a LOW /Disedisee: ettacr Ont ire@fien 2ANEF 14 MeReesary,) 
Reference the performance requirements summary Exhibit 12. The contract requires 
that taxi service meet a four minute response time with an accestable quality level 
of 15%. Rancom cbhservations indicate that this standard was not met. Ina sample 


of <0 dispatches, “0 dickups exceeded the standard.. 
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Contracting Officer) 
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During August, there was a limited number of drivers and vehicles avaflable due to 
sickness and maintenance. I will initiate short morning coordinaticn meetings each 
day at 7:30 a.m. so that maintenance and ocerations personnel can assure that 

enough drivers and vehicles are available fcr daily activities. 
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The contractor's actions should prevent further recurrence. A deduction of $¢é750 will 
be made from the August invoice computed as fotlows: Monthly Cost - $90,CCC 
Deduct Percentage 15% 
Percent of Samole Serective 0% 
ecuction 0/59 
If this recurs next month a deducticn will be taken plus a cure rotice will be issued. 
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a. Ground rules. This section lists the normal steps to 
be taken by contract administration when the QAE reports 
these deficiencies. The actions listed are not hard-and-fast 
rules, and are a minimum. More serious action can be taken 
sooner. 


(1) When the contractor's performance is 
unsatisfactory as defined in the surveillance plan and a 
formal action is indicated, the QAE, the functional area 
chief, and the contract administrator meet to determine what 
action is appropriate for the specific circumstances. 


(2) If a decision is reached not to take a monetary 
deduction, the reasons are documented. The contracting 
officer must indicate agreement with the decision by signing 
the contract discrepancy report or other decision 
documentation. 


b. Actions. Following are the actions normally taken 
when poor performance is found. 


(1) As a rule, the QAE tells the contréctor's site 
manager, in person, when discrepancies occur and asks the 
contractor to correct the problem. The QAE makes a notation 
On the tally or surveillance checklist, of the date and time 
the deficiency was discovered, and has a contractor 
representative initial the entry on the checklist. 


(2) If the number of discrepancies found exceeds 
the level for satisfactory performance, the QAE uses the 
decision tables in the surveillance plan to determine the 
cause(s). 


(a) If the government created any of the 
discrepancies, these are not to be counted against the 
contractor's performance. 


{b) When the government has caused the 
contractor to perform in an unsatisfactory manner, the QAE 
prepares a letter to be sent to the responsible organization 
requesting corrective action be taken. The QAE sends it to 
the organization through the contracting officer. 


(3) When the contractor is responsible for exceeding 
the limits of satisfactory performance, the contracting 
officer issues a contract discrepancy report (CDR) to the 
contractor (see paragraph 5-3c). If the failure is serious 
enough, issue the CDR at the time of the unsatisfactory 
performance, rather than at the end of the month. 
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(4) When a CDR is issued for a specific service 
the contracting office deducts from the month's payment, an 
amount up to the percentage indicated in the Performance 
Requirement Summary exhibit of the contract. Do not delay 
the deduction until the contractor responds to the CDR. If 
surveillance was done right and the decision tables used, the 
unsatisfactory performance is clearly the fault of the 
contractor. For a specific example of a deduction, see 
Paragraph 5-5. 


(5) If the contractor does not achieve satisfactory 
performance in that specific service by the end of the next 
month, the contracting officer issues another CDR and deducts 
the appropriate amount from the contractor's payment. 


(6) If a third CDR must be issued, consider issuing 
a cure notice. (However, a cure notice can be issued sooner, 
1f necessary). 


(7) Depending on the contractor's overall 
performance, the government may issue a Show Cause letter if 
the reply to a cure notice is unsatisfactory; next consider 
terminating the contract. 


5-5. Deductions For Non=-Performance. Through the Inspection 
of Services clause, the government can deduct from a 
contractor's payment an amount equal to the services not 
provided. 


a. To do this, the contract administrator must know the 
major cost categories in the contract and the percentage of 
cost each service output represents. The percentage cost of 
each service is found in deduct analysis; see chapter 2, 
paragraph 2-9. An example of how the deduct formula works is 
shown in figure 5-3. 


b. Suppose the bid schedule showed the monthly contract 
Price for vehicle operations, maintenance, and analysis as 
snown. The percentage cost of the service output is then 
found by looking at the Performance Requirements Summary 
Technical Exhibit in the contract statement of work. In the 
example, the percentage cost of quality cf completed work is 
10 percent. This is then multiplied by $100,000 to obtain 
the maximum amount to deduct. 


c. If completed work was unsatisfactory during the 
month (that is, did not meet performance values) and the 
percent of the sample found bad was 20 percent, $2000 would 
be deducted from the payment normally due the contractor. 
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Deduct Formula (Example) 


Le: Quality of completed work is unsatisfactory 
(AQL of 6.5% exceeded) . 
and: Contract price is $100,000 per month 
and: Quality of completed work deduct percentage is 10% 
and: Sample size is 50 
and: Number of defects in the sample is 10 (Reject number is 8) 
Then: Deduction from the current month's invoice is: 
Contract orice = $100,000 
X Deduct percentage = 10 
910,000 
X Percent of sample defective .20 
Deduction > $ 2,000 


Figure 5-3. Deducting for Non-Performance. 
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d. This amount for quality of completed work is deducted 
because the contractor failed to provide reliable, uniform 
services within the assigned performance values. Although 
some completed work may have met the standard during the 
month, the acceptable quality level was not met and at least 
20 percent of the observations were defective. Hence, the 
total quality performance requirement has not been achieved. 
As a consequence, the service output is unsatisfactory. 


9-6. Good Performance. When a contractor's quality control 
program works, good performance results. If the result of a 
QAE's surveillance shows consistently good performance, the 
amount of surveillance can be decreased. 


a. Reduced Inspection. Inspection can be reduced 
when the following conditions have been met for a sampling 
guide. 


(1) The preceding 4 lots (that is, the last 4 
months) have all been acceptable. 


(2) The number of defects in each of the preceding 
4 lots is less than one half of the acceptance number. For 
example, with an AQL of 6.5 percent and a sample size of 32, 
the acceptance number is 5. If two or less defects were 
found in each of the last 4 lots, reduced inspection could be 
used. 


(3) The normal sample size is being used. 


(4) The functional area chief and the contract 
administrator agree to use reduced inspection. 


b. Reduced Sample Size and Acceptance or Rejection 
Numbers. Reduced inspection decreases the sample size as 
shown in figure 5-4. In addition, the acceptance and 
rejection numbers change as shown in figure 5-5. To make the 
changes to the existing sampling guide, take the following 
steps. 


(1) Make sure that the original sampling guide was 
uSing the normal sample size. To determine this, see Chapter 
4, figure 4-3 and compare the lot size with the sample size 
in the sampling guide. 


(2) Find the new sample size by using figure 5-4. 
Take the lot size and find the new reduced sample size. 


(3) Using the AQL in the sampling guide and the new 
reduced sample size, see figure 5-5 for the new acceptance 








Lot Size Normal Sample Size Reduced Sample Size 





2-8 2 2 
9-15 3 2 
16-25 3 2 
26-50 8 3 
51-90 13 5 
91-150 20 8 
151-280 _.. a eae 32: 13 
281-500 50 20 
501-1, 200 80 32 
1.201-3, 200 125 50 
3.201-10, 000 200 80 
10,001-35 ,000 315 125 
36 ,001-150,000 500 200 
150,001-500,000 800 Sid 
500,001 and over _ 1250 ° 900 


Figure 5-4. Reduced Sample Size. 





Single sampling plans for reduced inspection 
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MIL-STD-105D Acceptance, Rejection Levels for Reduced Inspection. 


Figure 5-5. 





and rejection numbers. Note that there is a gap between the 
acceptance and rejection numbers (for example, sample size 32 
and AQL 6.5 percent, accept is 5 and reject is 8). This means 
that the lot would not be rejected unless 8 defects were 

found and would be accepted if 5 or less defects were found. 
However, a number of defects greater than five will be cause 
for returning to normal inspection (that is, return to the 
sample size and acceptance and rejection numbers used in the 
Original sampling guide). 


c. Returning to Normal Inspection. When reduced 
inspection is in effect return to normal inspection the next 
month under the following conditions. 


(1) When the number of defects exceeds the 
acceptance number under reduced sampling or, 


(2) The functional area chief and the contract 
administrator deem it necessary to return to normal 
inspection. 


d. Returning to Reduced Inspection. If during the 
first month of the return to normal inspection, the number of 
defects found is again less than 50 percent of the reject 
level, a return to reduced inspection may be done the next 
month. If the number of defects found is over 50 percent, 
then normal sampling must be accomplished until 4 months of 
less than 50 percent of reject level defects are found. 


5-7. Documentation. During the course of the contract the 
QAE retains a copy of all inspection schedules, tally 
checklists, and surveillance activity checklists. At the end 
of the contract period, the QAE forwards these records for 
inclusion in the contract file. However, when a specific 
service becomes unsatisfactory during a surveillance period, 
the inspection documentation supporting the contract 
discrepancy report is forwarded to the contracting officer no 
later than 5 working days after the end of the previous 
performance period. 
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APPENDIX D 
EXCERETS FROM NAVFAC 40-327 


This appendix includes various portions of NAVFAC 40-327 


entitled wservice Contracts : Specifications and 
emeveiliance." The first illustration, the Performance 
Requirements Summary (PRS), 1S a key and ccmpzreéhansive 
Surveillance dccument. ie Jistsmeea cum cen “OF werk tc be 
E-econgm-d Under the contract, eee required standard of 
performance, the methed of inspection for the item, and ths 


acceptable quaiity level (or allowable deviation). 

The next series of illustrations is Chapter 5, Quality 
Pesucance Methods. Each style of inspecticn, pe it planned, 
Bengoms; etc., 1s explained. PLP ericmaGrwenocsing. an e@poLo= 
priate method is provided. Sample service inventcry work- 
sheets and surveillance schedules are presented. 

The next Da cor this appendix is eha pra: 6, 
Surveillarce. Administrative responsibilities and precedures 
are listed. 

Appendix C is an abbreviated sample quality assurance 
plana for industrial sclid waste collection. Appendix D is an 
Peeauceicn f£OL £EhS utilization of random numbe= tables. 
Finally, current NAVFAC lot and sample tables are provided 


althecuch these will te modified by new ONR sampling tables. 


Lies! 





TASK 


Residential waste 
collection (Sect. 
00005, Clause 3.2) 


Scheduled industrial 


waste collection 
(Sect. 00005,. 
Clause 3.3) 


Unscheduled 
industrial waste 
collection (Sect. 
00005, Clause 3.4) 


Ash collection 
(Sect. 00005, 
Clause 3.5) 


ee oe 


Quality control 
program (Sect. 
00004, Clause 7) 


5 


STANDARD & 
INDICATOR 


-collection on day 
specified 
-no debris 


-cans left at curb, 


off street 


-collected within 
4 hr of specified 
time 

-no debris 


“containers properly 


repositioned 


-collected within 
12 hr of request 
for pick-up 


-collected within 
12 hr of request 
for pick-up 


-OC program 
established 

-activity in QC 
files 

-%C identified 
problems resolved 
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FIGURE 4-1 


METHOD OF 
SURVEILLANCE 


-planned sampling 
(primary) 

-validated complaints 
(secondary) 


-random sampling 


(primary) 
-validated complaints 
(secondary) 


-validated complaints 


-~validated complaints 


-planned sampling 
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Performance Requirements Summary Table 


(Sample) 
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AQL 


3 units 
per 100 


103% 


5 per 


month 


3 per 
month 
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CHAPTER 5: QUALITY ASSURANCE METHODS 


5. GENERAL. This section describes quality assurance evaluation 
methods used to monitor Contractor performance on service contracts. Before 
proceeding, the purpose and goals of QA should be discussed. 


a. When the Government purchases goods or services, there must be 
some means provided to attest to the value received for monies spent. To do 
this, the Government must be able to confirm that the quantity and quality 
of goods or services received conforms to contract requirements. 


b. The recipients of the contracted goods or services, in this case 
Naval shore activities, are responsible for developing and implementing 
procedures that assure that the Government is getting what it contracted 
for. These procedures are referred to as QA. 


c. Contractors, on the other hand, are responsible for providing 
Quality Control (QC). The purpose of OC is to control the service producing 
process, and to insure that the desired level of output quality is 
maintained. 


Sau QA PROGRAM. 


5.1.1 Traditional Approach. The Navy's traditional approach to 
surveillance of Service Contracts, often a hit-or-miss affair with no 


written plan, has not provided adequate quality assurance. 


a. The method of surveillance which is claimed to be uSed most 
frequently is 100 percent inspection. In reality, however, the inspection 
is often much less than total, since 100 hundred percent inspection is very 
costly and not always feasible. 


b. Further, traditional surveillance methods have usually focused 
on the work process (adherence to specified steps and frequencies) rather 


than on the quality of contract outputs. The net result does not assure 
Satisfactory quality performance. 


ae. 2 New QA Approach. The new QA approach, based on a written plan, is 
keved to performance oriented specifications. 


a. It focuses on the quality of the product delivered by the 
Contractor and not on the steps taken or procedures used to provide that 
PECaUCE. 


b. It includes appropriate use of preplanned inspections, 
validation of complaints, and unscheduled inspections. 


c. It provides a structured approach to surveillance that permits 
Management control of QA. 


S.1.3 Criteria. There are several criteria for good QA. 


a. First, the PWS must be written so that the quantity and quality 
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of required work outputs are measurable. The development of the PWS and the 
QA should be viewed as a single process. These documents are interrelated; 


One defines required work outputs and quality standards while the other 
defines how work outputs will be observed and measured. 


b. QA must provide for adequate and affordable contract 
Surveillance. The depth and detail of surveillance should be geared to the 
relative importance of the services monitored. 


c. QA must have the potential to support corrective action taken by 
the SCM/ROIC when non-performance or unsatisfactory performance occurs. 


d. There are three key ideas that are the basis for contract 
surveillance. 


(1) Outputs. Quality Assurance evaluates the output service 
provided by the Contractor. The output service can result either from a 
Contractor developed procedure or from a Government specified procedure. 
When the output is based on a Contractor developed procedure, the procedure 
is examined on an exceptions basis; that is, satisfactory service output as 
specified in the contract normally indicates that the Contractor is using 
Satisfactory procedures. The Government should be concerned with Contractor 
procedures only when services are not adequate. When the procedure is 
specified by the Government, compliance with the procedure is the desired 
output service. 


(2) Compliance. Contractor's compliance with contract 
requirements is monitored through the performance indicators and standards 
which are specified in the PWS. Performance indicators are measurable 
attributes of the outputs. A standard is the gauge that Contractor 
performance is compared against. For example, scheduled trash collection is 
the work required, one of the indicators of good performance is timeliness, 
and the standard is trash pick-up is made within 4 hours of the scheduled 
time. 


(3) Cause of Problem. When observed performance indicators 
show output not to be in compliance with contract requirements, the QAE 
identifies the cause of the problem. The QAE looks beyond service outputs 
to determine if the problem is caused by the Government or the Contractor. 
If the cause of the problem rests with the Government, corrective action 
must be taken through Government channels and no action is required vf tiie 
Contractor. If the Contractor is at fault, the Contractor is told to take 
corrective action; payments to the Contractor are reduced; and a Contracts 
Discrepancy Report (CDR) may be issued. 


5.2 METHODS OF SURVEILLANCE. There are five methods that may be used 
for contract surveillance. 


ele: One Hundred Percent Inspection. One hundred percent inspection 
requires that output from each and every work occurrence be evaluated. One 
hundred percent inspection measures the Contractor's true level of 
performance. It is an expensive and time consuming method which should be 
used sparingly. 
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5.2.2 Planned Sampling. Surveillance by planned sampling is designed to 
evaluate a part but not all of a contract requirement. 


a. The number of inspections and the items to be inspected are 
based on subjective judgment. 


b. Planned sampling is useful when requirements at one location are 
more important than those at other locations; for example, galley dumpsters 
as Opposed to those in a remote administrative area. It is also useful when 
the Contractor's performance is poor in some spots but better in others. 


ae2s3 Random Sampling. Surveillance based on random sampling evaluates 
part but not all of the work performed. (Appendix D provides details of the 
mechanics of random sampling.) 


a. Using random sampling, any occurrence of work is as likely to be 
monitored as any other occurrence. The QAE's bias does not affect the 
specific occurrences of work selected for evaluation since all occurrences 
of an item of work are assumed to have the same level of importance. 


b. This method estimates the Contractor's overall level of 
performance for a given contract requirement. It is most useful when 
evaluating items that are repetitive in nature such as janitorial work, 
grounds maintenance, or service call work. 


c. To achieve full benefits, the random sampling method must be 
applied properly. If misapplied, results will be biased. 


56264 Validated Complaints. Validated customer complaints constitute a 
surveillance method based on customer awareness. Customers, familiar with 
contract requirements, notify the QAE when there is a case of poor or non- 
performance. Upon notification, the QAE investigates the report and, if 
valid, documents the performance problen. 


a. Formal customer complaints are a means of documenting certain 
kinds of service problems. The way to obtain and document customer 
complaints needs to be carefully planned by the persons monitoring the 
service contract. 


6b. Customer complaints are not random, but when validated by the 
QAE may be used to deduct money from the Contractor. 


c. When random sampling is the chosen method of surveillance, a 
customer complaint cannot be used to Satisfy a random observance. However, 
it can be used as further evidence of unsatisfactory performance if random 
sampling shows that the specific service is unsatisfactory. These 
complaints can be used to decide if other action should be taken. 


(1) Getting Customer Complaints. An aggressive customer 
complaint program, once established, needs to be explained to every 
organization that receives the Contractor's services. An operating 
instruction should be given to each organization outlining the customer 
complaint program, the format and the content of a formal customer 
complaint, and the action which can be expected from those assigned to 
watching and managing the service contract. (Appendix E provides a 
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Customer's Guide for Evaluating Contractor Performance.) 


(2) Documenting the Customer Complaint. Normally, each 
customer complaint is brought, in person or by telephone, to the individual 
checking contract performance. Enter complaint information into a Customer 
Complaint Record, similar to the sample show in Figure 5-l. The record 
contains the following information: 


i Date and time of complaint. 

2 Source of complaint - organization or individual. 
3 Nature of complaint (narrative description). 

4 Contract reference of complaint related services. 
5 Valid complaint (Yes or No). 

6 Date Contractor informed of complaint. 

7 Action taken by Contractor. 


8 Signature of the person receiving and validating the 
complaint. 


5.2.5 Unscheduled Inspections. The QAE may conduct impromptu evaluations 
Of contract requirements whenever necessary. This surveillance method 
(which is not really a method at all) provides no information on the 
Contractor's overall performance. 


5.2.6 Criteria for Method Selection. No firm guidance for method 
selection can be provided in a manual of this type. Some general guidance 
on selection criteria are: 


a. Population size refers to the number of scheduled or expected 
occurrences of a contract requirement over a given time period, usually one 
montn. The actual number of occurrences will depend on how a unit of 
Service is defined. Frequency of service at any location may be daily, 
weekly, etc. Population size is easy to determine for scheduled services. 
When services are performed on an "as required" basis, population size must 
be estimated based on historical or projected data. Large homogeneous 
populations are ideally suited to random sampling. 


b. Some contract requirements are more important than others. Some 
requirements may have an impact on an activity's mission. Others have 
littie or no impact. One hundred percent inspection might be used for 
"important" contract requirements, a sampling method or customer complaints 
for "less important requirements", and validated complaints for the "least 
important requirements”. 


c. It is easier to talk about instances where surveillance methods 
are not appropriate. 





(1) One-hundred percent inspection is not suited to large 
populations. It would be time consuming for the QAE and expensive to the 
Government. 


(2) Those services that are important or costly require tight 
surveillance. Validated complaints do not guarantee that all instances of 
nonper formance or poor performance will be reported in a timely manner. 


(3) If a service is required but individual occurrences are of 
small importance (for example, emptying a trash can), it is normally not 
beneficial to invest a great amount in surveillance. One-hundred percent 


inspection is the most costly of the evaluation methods and should not be 
used. 


(4) If a contract requirement is continuous in nature, 100 
percent inspection is not feasible since it would require a QAE to be 
on-site full time. Examples of continuous requirements are manning a guard 


post, maintaining a minimum inventory of carts, and fulfilling contractor 
quality control requirements. 


(5) It is not possible to use a surveillance method that 
requires prescheduling of evaluations for unscheduled service such as 
responding to Emergency/Service (E/S) calls, processing work orders, and 
dispatching vehicles. It is possible to schedule retrospective evaluation 
Of Management Information Systems (MIS) outputs such as logs, work orders, 
Or other written records. 


d. The choice between planned sampling and random sampling is 
sometimes difficult. 


(1) For example, on a Bus Service Contract it would be very 
easy to establish a random sampling evaluation plan where the QAE monitors 
the Contractor's compliance with the established bus schedule. However, 
this type of surveillance plan would require the QAE to travel to all areas 
of the bus activity on a random basis, resulting in a large amount of 
unproductive QAE travel time. Because of the many customer complaints that 
would result if there were poor bus service, planned sampling is a more 
attractive surveillance method. 


(2) A good rule of thumb in choosing between random and planned 
Sampling is that if all evaluation can be conducted at one work site, random 
sampling is preferred. If work sites are dispersed, planned sampling should 
be used. On the other hand, increased QAE travel time may be a good 
investment if the work item is important. 


e. Ideally, QAE's should be staffed to the level required to 
Support the selected QA Program. In reality, the QA program used must 
accommodate the availability of QAE's. A combination of QA methods should 
pe considered to get the best QA program possible with a given number of 
QAE's. 


5.2.7 Outputs Subject to Surveillance. The QAE must determine what output 
to inspect to determine the Contractor's overall performance. In many 
instances a contract requirement will generate several outputs. Evaluating 
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DATE: TIME: 


SOURCE OF COMPLAINT: 
Organization - 


Individual - 


NATURE OF COMPLAINT: 


CONTRACT REFERENCE: 


COMPLAINT VALIDATED: 
Date - Time - By - 
CONTRACTOR INFORMED OF COMPLAINT: 
Date - Time - By - 


ACTION TAKEN BY CONTRACTOR: 





WORK REINSPECTED: 





Date - Time - By - 
Satisfactory - 
Unsatisfactory - 

FIGURE 5-1 


Customer Complaint Record 





a Single output may provide a reliable indication of the Contractor's 
overall performance. Types of outputs subject to inspection are: 


a. Work Performed. Inspection of all work performed is the best 
way to evaluate Contractor performance. This requires that the QAE be at 
the work site during performance of the work, or be there shortly thereafter 
if results of work performed are visible (e.g., janitorial, grounds, etc.). 


b. Records. If the QAE is unable to be at the work site at 
required times, the inspection of records, work chits, and other documents, 
(combined with spot checks of actual work performance,) may provide a 
satisfactory indication of work performance. 


c. Management Information System (MIS). MISs usually collect 
information over a specified period of time. This information can be 
compared to a contract standard. On the basis of this comparison, 
performance can be evaluated and the performance for the specified period 
can be accepted or rejected. 


(1) For example, the vehicle down time rate is computed every 
month. A simple comparison of the rate with the maximum acceptable rate in 
the SOW explains a great deal about the level of maintenance service 
supporting the base vehicles. 


(2) By way of caution, one is going to use a MIS to check the 
Contractor, make sure the MIS contains reliable data. 


a.2.8 Inventory of Services. Each service requirement that is to be 
monitored must have an Inventory of Services worksheet prepared. Figure 5-2 
is a sample of an inventory worksheet. Inventory worksheets are prepared at 
the same time the QA Plans are prepared. 


a. Purpose. Inventory of Services Worksheets serve two purposes. 
Pirst and most obvious, these worksheets provide a comprehensive listing of 
locations receiving a given service. Second, these worksheets are used to 
select locations for inspection when one of the sampling methods is used. 


b. Numbering. Worksheets list each location where a service will 
be verformed. Work occurences should be grouped as daily, weekly, or 
monthiy depending on how surveillance is to be conducted (1.e., daily, 
weekly, monthly, or other.) Each work occurence, within the group, should 
then be sequentially numbered. 


ae 3 QA PLANS. The most important part of implementating a QA program is 
the development of comprehensive QA Plans. (Appendix C provides a sample of 
a QA Plan for Scheduled Solid Waste Collection.) 


a. QA Plans are documentation of how the QAE intends to monitor 
Specific aspects of Contractor's performance. 


b. These plans are the basis for developing QAE schedules and 
determining required QAE staffing levels. 








INVENTORY OF SERVICES FOR: Industrial/Administrative Solid Waste Collection 
SURVEILLANCE SCHEDULE: Weekly 


NUMBER : LOCATION/IDENTIFICATION : NUMBER : LOCATION/IDENTIFICATION 
1 2 BLDG 1 1 : 2 : 
2 : BLDG 1 1 $ - : 
3 $ BLDG 2 : : : 
4 : BLDG 3 $ : : 
5 : BLDG 4 l North End : $ : 
6 : BLDG 4 2 North End 3 : 3 
7 : BUDG 4 3 South End : 5 $ 
8 : BLDG 5 : 3 . 
9 : PARK ADMIN BLDG : : : 

+ $ PARK MAINT BLDG : : : 
ll : : : : 
12 2 : : : 
13 : : : : 
14 : : : : 
25 : " : : 
16 $ I $ : 
Le : : : $ 
18 : : $ ; 
$ : : : 
FIGURE 5-2 
Inventory of Services Worksheet 
(Sample) 


c. QA Plans are developed to cover all items shown in the 
Performance Requirement Summary Table. (A single plan may cover more than 
one item if surveillance of those items is compatible.) 


d. QA Plans are monitored by the QAE throughout the life of the 
contract. Once established, they do not tend to change. 


e. Recommended formats for QA Plans are described below. (A sample 
of a OA Plan is provided in Appendix C.) 


f. Each QA Plan established should have an evaluation worksheet, or 
checklist. This worksheet, or checklist, is used to document surveillance 
results. Figure 5-3 is a sample of an Evaluation Worksheet for refuse 
€O ection. 


Seo. k 100 Percent Inspection. 


a. Contract Requirement. Contract requirements, identified in the 
Performance Requirements Summary Table (see Figure 4-1), are listed along 
with the performance indicators and standards. Specific clauses in the SOW 
establishing these requirements also are listed. 


bt 
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SURVEILLANCE SCHEDULE FOR INDUSTRIAL/ADMINSISTRATIVE AREA SOLID WASTE COLLECTION 
CONTRACT NUMBER: 





FOR: 22 June 1981 QAE: I. Peak 






































BLDG/ 
LOCATION EMPTIED DEBRIS POSITION CONDITION COMMENT 
Bldg. $ : : : : 
$14 $ P : P $ P : P : 
Blidg. : : : ; :Trash sovilled on pavement 
#19 : P ; FP : P : P swhen dumoster emptied 
Bldg. : ; : 
#28 : P $ P : P : P ; 
Bldg. : : ; : sContainer overflowing, 
$41 : F : P : P $ P :scheduled pick-up yesterda 
Bldg. : : : : 
#43 $ P : P : P : P : 
Area ; : : sContainer lid will not 
# 3 : P : P : P : F sclose 
Area : $ : : : 
#11 : P P P : P : 
Area : : $ : : 
319 : P P : P : P : 
Area ~ : : 
#23 : P : P $ P : P : 
FIGURE 5-3 


Evaluation Worksheet for Scheduled 
Industrial/Administrative Area Solid Waste Collection 
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b. Primary Method of Surveillance. The primary method of 
evaluation is 100 percent insvection. 


¢c. AQL. AQL'S are best stated as a percentage 
dq, Quantity of Work. Define a unit (i.e., single occurrence) of 


work, and determine the number of units to be performed during the 
surveillance period. Performance of each unit will be evaluated. 





e. Level of Evaluation. This is not applicable 

£. Sample Size. This is not apolicable. 

g. Sample Selection Procedure. This is not applicable 

h. Evaluation Procedure. List any procedures or checklists us 
when doing the inspection. Evaluation procedures should be in eneugh 4 


to allow others to continue the same manner of inspection using the same 
evaluation criterta. 
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i. Analysis of Results. The analvsis of evaluation results for 100 
percent inspection is straight forward. The Observed Defect Rate (ODR) is 
computed as follows: 


Number of Defects 
— Number Units of Work es 


Se 5c Planned Sampling. 


a. Contract Requirement. Contract requirements, identi 
Performance Requirements Summary Table (see Figure 4-1), are lis 
with the performance indicators and standards. Specific clauses in the SOW 
establishing these requirement are listed. 


b. Primary Method of Surveillance. The orimary method of 
surveillance is planned sampling. Validated complaints are a good 
Supportive surveillance method. . 


c. AQL. AQL's should be stated as an absolute value (e.q. 3 ver 
surveillance period). 


d. Quantity of Work. Define a whet (i.e. gies occurrence) of 
work and determine the number of units to be performed during the 
surveillance period. 


e. Level of Surveillance. I£ more shan one level of surveillance 
is desired, it is recommended that three carefully defined levels of 
surveillance be identified and that criteria be establisned for switchin 
from one level to another. These surveillance levels are: 


We) 


Normal - Applied to good but not exceptional contractor 
performance. tais level of surveillance to be used 
when contract is first implemented. 
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Reduced - Applied in the case of exceptional contractor 
per formance. 


Increased - Applied in the case of poor contractor performance. 


f. Sample Size. Determination of sample size for planned sampling 
is subjective. In order to provide consistent surveillance, the rationale 
for selecting a sample size must be identified. Sample size will vary 
depending on the level of surveillance used. 


¢. Samole Selection Criteria. The criteria for sample selection 
must be Gocumented and applied consistently from surveillance period to 
surveillance period. If there is no consistency, trends in contractor 
performance cannot be detected. 


h. Evaluation Procedure. List any procedures or checklists used 
when doing inspection. Evaluation procedures should be in enough detail to 
allow others +o continue the same manner of inspection using the same 
evaluation criteria. 


i. Analysis of Results. The Observed Defect Rate (ODR) for a 
planned inspection is the total number of defects documented during the 
Surveillance period. All defects detected by customer complaints and 
unschedule insdections are included in this total. The ODR is: 


ODR = number of decumented defects. 


Ses. 3 Random Sampling. This method, in order to be effective, must be 
properly applied. (Appendix D describes the mechanics of random sampling.) 


a. Contract Requirement. Contract requirements, identified in the 
Performance Requirements Summary Table (see Figure 4i-1l), are listed along 
with the performance indicators and standards. Specific clauses in the SOW 
establishing these requirements are listed. 


b. Primary Method of Surveillance. Random sampling is the primary 
evaluation method. Validated complaints and unscheduled inspections may be 
considered as secondary methods. WARNING information collected by other 
Surveillance methods can never be combined with information gathered by 
sandom Sampling. Evaluation results coilecte’d by Sther Mathods serve cntiy 
aS supportive data. 


c. AQL. For evaluation by random sampling, AQL's are stated as a 
percentase and oredefined as 5, 10, 15, 20 or 25 percent. Sample sizes for 
AQL's of 15, 20 and 25 percent are defined in the Sample Size Tables but are 
not generally recommended. 


d. Quantity of Work Performed. Define a unit of output for each 
Service that is subject to inspection. It is important to accurately 
determine the quantity of work performed in order to select the appropriate 
sample size. 





e. Level of Surveillance. The level of surveillance will be set at 
one of three levels, and the level of surveillance may be adjusted monthly 
depending on the Contractor's performance. 


(1) Initial level of surveillance is normal surveillance, level 
II. 


(2) Reduced Surveillance. If contractor performance has been 
"excellent," the level of surveillance could be reduced to level I. 


(3) Increased surveillance. If, on the other hand, performance 
during the past surveillance period was poor, surveillance could be 
increased to level III. 


f£. Sample Size. Sample size for random sampling is determined by 
use of tables and is a function of the AQL, quantity of work performed and 
level of surveillance. (Sample size table are provided in Appendix D.) 


g. Sampling Procedure. To assure that samples are selected 
completely at random, a random number table must be used. (This table is 
provided as part of Appendix E.) ; 

h. Evaluation Process. List any procedures or checklists used when 
Going the inspection. Evaluation procedures should be in enough detail to 
allow others to continue the same manner of inspection using the same 
evaluation criteria. 


i. Analysis of Results. Analysis for random sampling will consist 
of computing the Observed Defect Rate (ODR) for the past surveillance period. 


Number of Defects 
sas Number Evaluations Conducted ee 


The number of defects used in computing the ODR is derived from 
the samping process. Defects detected through validated complaints or 
unscheduled inspections cannot be used. 


5. 35.M Validated Customer Complaints A Validated Complaint is any customer 
complaint identifying a Contractor defect that the QAE has validated by 
documentation based on an on-site visit. Complaints not validated may not 
be used. 


a. Contract Requirement. Contract requirements, identified in the 
Performance Requirements Summary Table (see Figure 4-1), are listed along 
with the performance indicators and standards. Specific clauses in the SOW 
establishing these requirements are listed. 


b. Primary Method of Surveillance. The primary method of 
surveillance is validated complaints. A secondary evaluation method that 
may be used is unscheduled inspections. 


c. AQL. AQL's for Validated Complaints should be stated in terms 
of number of occurrences per surveillance period. 





d. Quantity of Work. Although the quantity of work will not have 
direct effect on validated complaint evaluations, this information is useful 
when putting results in perspective. 


e. Level of Surveillance. This is not applicable. 
£. Sample Size. Tnis is not applicable. 
g. Sample Selection Criteria. This is not applicable. 


h. Evaluation Procedure. Document how validation of complaints is 
to be performed. Evaluation procedures should be in enough detail to allow 


others to continue the same manner of inspection using the same evaluation 
criteria. 


i. Analysis of Results. Determine the number of validated 
complaints for the past surveillance period. If there is a good customer 
complaint program, changes in the number of complaints per surveillance 
period may be useful in detecting changes in the Contractor's overall level 
Of performance. The Observed Defect Rate (ODR) for validated complaints is: 


ODR = number of documented defects 


De Unscheduled Inspections. Unscheduled inspections do not have formal 
QA Plans. This type of surveillance should be used as a secondary, or 


supportive, surveillance method and as such, be subject to the QA Plan of 
the primary method. 
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CHAPTER 6 SURVEILLANCE 


6. GENERAL. The key to assuring satisfactory performance from service 
contracts 1s adequate Government surveillance of Contractor performance. 
Hit-or-miss surveillance by untrained personnel is an invitation to poor 
performance. The more prone (historically) a particular type of work is to 
shoddy performance, the more necessary it is to assign an adequate number of 
trained and qualified personnel (QAE's) who are familiar with the contract 
surveillance. The QAE is a key person in service contract management. The 
QAE serves as the eyes and ears of the SCM and as such must demonstrate a 
large cegree of common sense since many facets of the job are subjective and 
Open to criticism. NAVFAC Quality Assurance Evaluator (QAE) Training 
Manual, MO-326.2, provides in-depth information on QAE duties and the 
surveillance process. 


6.1 SURVEILLANCE PROCESS. The surveillance process is a system composed 
of many key elements. Figure 6-1 illustrates this system. 


Geis lL Inputs. The key input to surveillance is contract requirements. 
These requirements dictate what work the Contractor is to perform and what 
the QAE is to evaluate. Next in importance is the Contractor's work 
schedule. The schedule is necessary in order for the QAE to know when work, 
which is not scheduled by contract requirements, is to be performed. The 
intensity of surveillance is influenced to a degree by the Contractor's past 
performance. During the surveillance period the number and type of customer 
complaints received will affect the QAE's schedule. 


Gok. 2 Process. The process, as displayed in Figure 6-1, may be divided 
into four main parts: planning for surveillance, conducting surveillance, 
analysis of surveillance results, and taking action based on those results. 


a. Planning for surveillance includes QA Plans and Monthly QAE 
Schedules. QA Plans are developed prior to contract award and in most cases 
remain unaltered throughout the life of the contract. Monthly QAE 
Surveillance Schedules are developed, based on QA Plans, at the start of 
each surveillance period (a period is usually one month). 


b. Contract surveillance is conducted in accordance with QA Plans 
and the QA Monthly Schedule. If, during the surveillance period, major 
discrepencies are noted (and documented), the QAR will alert the SCM that 
action should be taken. 


c. At the end of the surveillance period, "documented" surveillance 
results are analyzed to determine the Contractor's overall level of 
per formance. 


dad. Based on the Contractor's performance, there are several courses 
of action that may be taken. First, deductions will be made for all 
observed and documented cases of non-compliance, regardless of the 
Contractor's overall level of performance. Other specific actions that may 
be taken include: “jaw boning" the Contractor; issuance of a Contract 
Discrepancy Report (CDR), Cure Notice or Show Cause Notice; and contract 
termination. The person taking action may be the SCM, ROIC, OIC, or 
Commander, NAVFAC. Regardless of the course of action, GOOD DOCUMENTATION 
IS REQUIRED. 
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FIGURE 6-1 
Contract Surveillance Process 
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6.2 QUALITY ASSURANCE EVALUATOR (QAE). QAE's must have qualifications 


in both the technical aspects of the contracted function and contract 
surveillance methods. 


a. Technical expertise for evaluating work quality is found within 
the organizational component that would be responsible for work 
accomplishment if it were to be performed by Government forces. The QAE 
should be drawn from this component. (In the case of CA conversions the 
activity will retain 4 percent of the organizational component being 
converted to contract performance. This 4 percent is intended for contract 
management ~- i.e. SCM and QAE duties.) 


be. The TRCO is the person usually responsible for selecting QAE's. 
Once selected for QAE duties, candidate QAE's must be trained in contract 
surveillance methods. This training is available through the QAE Training 
Course offered periodically by NAVFAC Engineering Field Divisions (EFD's). 


6.2.1 Authority. QAE derives his authority by delegation from the SCM and 
acts as his representative when authorized. In no case can the SCM delegate 
authority to the QAE that he himself does not have. 


a. Delegation of authority is by letter from the SCM to the QAE,. 


b. The QAE has no authority to allow deviations from essential 
contract requirements, but when authorized by his superior, he may approve 
minor deviations not involving change in contract time, price, or basic 
design. 


c. The QAE has no authority to direct or interfere with the methods 
of performance by the Contractor or to issue instructions directly to any of 
the Contractor's personnel unless the methods being used are unsafe. 


Gaizee Responsibilities. The QAE, because of his familiarity with the 
contract, the Contractor, and the customer, is involved with several aspects 
of Service Contracting. 


a. Specific QAE responsibilities will depend upon local conditions, 
size of the contract, QAE collateral duties, etc. Assigned responsibilities 
should be agreed upon by the SCM, the QAE, the TRCO, and other interested 


parties pefore the contract start date. Assigned QA responsibilities shoniad 
be stated in writing. 


b. Specific QAE duties are to: 


oO Review plans and specifications prior to IFB/RFP; 

° Conduct Contractor pre-bid site visits; 

oO Assist in pre-award surveys; 

° Review Contractor schedules and advise SCM of acceptability; 
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° Attend pre-bid and pre-start conferences; 


° Coordinate provision of Government furnished space, utili- 
ties, equipment and material; 


° Prepare Quality Assurance Plans; 

° Prepare surveillance schedules; 

° Perform surveillance and document Contractor's performance; 

° Conduct surveillance on accomplishment of re-performed work; 

oO Monitor labor, safety and security practices, and document 
results; 

° Recommend payment deductions for unsatisfactory work to 
SCM; and 

° Assist in the preparation of Government estimate for change 
orders. 


6.2.3 STAFFING. An adequate level of QAE staffing is required to make any 
QA program work well. There are two approaches to staffing for contract 
Surveillance: (a) write the QA program to accommodate the number of QAE's 
currently available, or (b) to write the QA program to provide the desired 
level of surveillance and staff to that program. Obviously the latter is 
the preferred approach. The chief problem is converting the desired level 
of surveillance into manhour requirements. QA Plans, and the subsequent 9AE 
Schedules, provide a means of determining QAE manhour requirements. These 
documents, if properly prepared, will identify QAE staffing requirements for 
contract surveillance. 


6.3 QAE SCHEDULE. The development of an effective evaluation schedule 
should be of the utmost importance to the QAE. The evaluation schedule 
allows the QAE to plot where he should be on any given day of the week. By 
developing a balanced inspection schedule, a QAE can be much more effective 
in his job. It allows the QAE to plan his workday in advance to the best 
advantage. It also allows him to utilize his time and eliminate some 
potentially wasteful actions (for example, excessive travel time between 
inspections). QAE Schedules serve three purposes,they are: 


a. Optimizing Time. The QAE will use his established schedule to 
plan his work. By making maximum use of a good schedule, the QAE will 
optimize use of his time. 


b. Management Control. The SCM is provided a copy of each QAE 
Schedule. The SCM has the responsibility to see that surveillance of 
service contracts is properly conducted. The QAE Schedule provides him the 
information necessary to monitor the QA program. 





c. Audit. The QAE Schedule, along with completed evaluation 
reports, provide an audit trail for contract surveillance. The established 


schedule as updated during execution, should reflect what was actually 
accomplished. 


6.5.1 Schedule Development. QAE schedules are based on established OA 
Plans. When developing monthly schedules, the QAE will use the evaluation 
worksheets developed for that month. As he prepares his schedule he may 
find it convenient to combine surveillance requirements in order to 
streamline the daily schedule. When developing schedules it may be 
necessary to modify OA Plans in order to achieve the most effective 
allocation of QAE time. However, plans based on random sampling CANNOT be 
modified in this manner. 


a. The Schedule. An example of a QAE's Schedule is shown in Figure 
6-2. This example shows only a 6-day schedule. The QAE must make up enough 
sheets to include each day of the month. Along the top of the schedule, 
insert the items to be checked during the month. Along the bottom of the 
schedule, indicate the number of observations to be made during the month. 


b. Filling In and Updating the Schedule. To fill in the inspection 
schedule, the QAE refers to the QA Plans for each service being monitored. 
The QA Plan is used to determine the inspections (observations) to be made 
during the month. 


(1) Contract surveillance must cover all hours of operation. 
Random observations are scheduled at night, on weekends, and on holidays 
when services are performed during these periods. Areas that are monitored 
on a set schedule are included in the monthly schedule. This monthly 
schedule shows where and what the QAE is monitoring at all times. 


(2) Post any changes to the schedule weekly, and send copies to 
the SCM and to the TRCO. Document and explain the reasons for each change. 
Actual surveillance activity recorded on the evaluation worksheets must be 
comparable to the monthly schedule. 


(3) As updated, one must be able to conduct a complete audit 
trail from the monthly schedule, to observing the QAE perform sampling, to 
completion of the evaluation worksheet. 


(4) After the schedule is completed and filled in, this form is 
"FOR OFFICIAL USE ONLY" and must not be shown to the Contractor. 


Cc. Review and Approval. The QAE's Schedule must be submitted to 
the SCM for review and approval. The QAE is responsible for posting 
changes, as they occur, to the schedule throughout the month. This schedule 
becomes a formal part of the surveillance documentation, and, as such, it 
must be auditable. 


6.4 SURVEILLANCE. Contract surveillance involves using the QA Plan 
called for in the monthly schedule. 
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6.4.1 Performance of Surveillance. Contract surveillance is performed in 
accordance with the QA Plan: method of surveillance, sample size/selection, 
and evaluation critieria are specified in this plan. 


a. The OA program must provide an adequate and consistent level of 
surveillance. It is important that QA plan be followed and deviations 
documented in order that this program can be audited. 


b. Timing of Inspections: 


(1) In some cases inspection will have to be conducted during 
the period of work performance. For example: 


- When the Contractor is performing maintenance on a piece 
of equipment, the QAE may have to inspect the work before the equipment is 
fully reassembled. 


- If work on a building's electrical or plumbing systems is 
performed, the QAE would have to inspect before siding or dry wall is 
replaced. 


- To determine if the Contractor is maintaining a base 
shuttle bus schedule, the QAE must be at the stop at the scheduled time. 


(2) For daily services, such as custodial services, the QAE 
should conduct inspections shortly after work performance, but prior to 
occupant use. 


(3) In many cases services performed will provide outputs of a 
lasting nature and may be inspected days after actual performance. Work, 
such aS painting, resurfacing of roads, glass replacement, tree pruning, 
etc., are examples of this type of work. 


(4) Some services performed by the Contractor may be 
inspectable at any time; for example: if a grounds maintenance contract 
requires a level of maintenance (vs. frequency of work), the QAE will be 
monitoring the condition of the grounds rather then work performed (e.q. 
grass to be between 2 and 4 inches in height.) A watch standing requirement 
Such as guard service or fire protection requires Contractor personnel on 
duty 24 hours a day. 


c. It is good practice to make surveillance findings, good or bad, 
available to the Contractor on a daily basis. Provided information does not 
relieve the Contractor of his Quality Assurance efforts but is intended to 
keep the Contractor advised of the Government's perception of the quality of 
performed work. 


6.4.2 Documentation. Just as services required of the Contractor have 
outputs, Government surveillance has outputs. One of the key outputs is 
documentation, which consists of: the QAE's monthly schedule, completed 
evaluation worksheets, records of customer complaints, and any other 
Material that reflects the quality/quantity of Contractor performance. 
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QAE: SCHEDULE FOR WEEK OF : June 22 thru 27 








DAY 
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: : : MONITOR : R-INSP 4 
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: swith SCM, : COLLECTION : 14,18,20 : TRASH : 
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¢ OC FILE :TO DISCUSS: : : AREA B : 
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1100: : : : : : 
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FIGURE 6-2 
QAE Surveillance Schedule 
(Sample) 
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a. Recording Observations. Evaluation Worksheets are used for each 
QA Plan. They are used to tally information on scheduled observations and 
noted defects. Each observation in the sample is recorded on the Worksneet, 
and the Worksheets then become a formal Government record. 


b. Potential Unsatisfactory Performance. If surveillance indicates 
that the number of defects for the month may exceed the AQL, the QAE should 
try to identify and document the cause of the problem. 


(1) The cause of the problem may lie with the Government. 


(a) Is delivery of Government-provided material or 
equipment the problem? 


(b) Are Government employees (civilian or military) 
disrupting the Contractor's work efforts? 


(2) If the cause of the problem lies with the Contractor, the 
QAE should determine why. 


(a) Does the Contractor have an adequate number of people, 
or properly trained people, at the work site? 


(b) Is Contractor work supervision adequate? 


(c) Is the Contractor's QC program identifying these 
problems? (It should be.) ‘ 


(d) Are the proper equipment and materials being used? 


(e) Is the work method used capable of producing the 
required output? 


c. During the course of the contract the QAE retains a copy of all 
QAE Schedules, Evaluation Worksheets, and checklists. At the end of the 
contract period, the QAE forwards these records for inclusion in the 
contract file. However, when a specific service becomes unsatisfactory 
during a surveillance period, a copy of the inspection documentation 
supporting the contract discrepancy is forwarded to the SCM/ROIC for action. 


d. As mentioned above it is good practice to keep the Contractor 
appraised of surveillance results. One way to do this is to provide the 
Contractor's Representative a copy of the Evaluation Worksheet. [It is 
recommended that Contractor's Representative initial the original Evaluation 
Worksheet indicating that he has received a copy. 


Gao SURVEILLANCE RESULTS. It is the QAE's duty to make OA evaluation 
results known to the SCM who then is responsible for taking the appropriate 
action. At the end of the month the QAE will tally the results for all 
Evaluation Worksheets, checklists, etc., to determine the Contractor's 
overall performance with respect to each QA Plan. 
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Be o<k Excellent Performance. If the Contractor has performed in the best 
possible manner and there were few defects noted (Observed Defect Rate (ODR) 
less than 1/2 the AQL), the QAE might suggest that: 


a. The Contractor should be notified by the SCM that he is 
performing satisfactorily. 


b. The level of surveillance might be reduced. 
c. Deductions will be made on all documented defects. 


Gas. 2 Good Performance. When a Contractor's quality control program 
works, good performance results. If the result of a QAE's surveillance 
shows consistently good performance, the amount of surveillance can be 
decreased. 


a. Deductions will be made on all documented defects. 


b. Reduced Surveillance. Inspection can be reduced when the 
following conditions have been met for a surveillance period. 


(1) The preceding month's work (or number of months as 
specified in the QA Plan) has been acceptable. 


(2) The percentage of defects in the preceding month(s) is less 
than one half of the AQL. 


(3) The normal sample size is being used. 
(4) The TRCO and the SCM agree to use reduced inspection. 


c. Returning to Normal Surveillance. When reduced surveillance is 
in effect, return to normal inspection the next month under the following 
conditions: 


(1) When the percentage of defects exceeds the AQL under 
reduced sampling, or 


(2) The TRCO and the SCM deem it necessary to return to normal 
inspection. 


6.5.3 Questionable Performance. An outcome of questionable per formance 
can only result when random sampling is the surveillance method used. 


a. Random sampling procedures take into consideration potential 
errors in results. Since random sampling only provides an estimate of the 
true defect rate, a margin for error must be used. This is done by 
specifying the accuracy desired of the ODR, as compared to the true defect 
rate. Accuracy is defined to be one half of the AQL. If, for example, the 
AQL was 10 percent, accuracy would be 5 percent. [It is this gray area, 
where the ODR falls between the AQL and 1.5 times the AQL, that results are 
questionable. 


b. The recommenced actions for questionable performance are: 
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(1) Deduct for all documented defects. 


(2) If there are a significant number of validated customer 
complaints and/or defects detected by unscheduled inspections, issue a 
Contract Discrepancy Report (CDR). 


(3) Go to an increased level of evaluation. 


6.5.4 Poor Performance. If the Contractor has displayed poor performance 
(i.e., the ODR exceeds the AQL, or for random sampling the ODR exceeds 1.5 
times the AQL), then the following are the actions normally taken in 
addition to deductions. 


a. If the QAE first determines that the Government created any of 
the discrepancies, these are not to be counted against the Contractor's 
performance. When the Government has caused the Contractor to perform in an 
unsatisfactory manner, the QAE prepares a letter to be sent to the 
responsible organization, requesting corrective action. The QAE sends it to 
the responsible organization through the scm. 


b. If the Government did not cause the discrepancy, the QAE tells 
the Contractor's site manager, in person, when discrepancies occur and asks 
the Contractor to correct the problem. The QAE makes a notation on the 
Evaluation Worksheet of the date and time the deficiency was discovered and 
has a Contractor's representative initial the entry on the checklist. 


c. Increased Surveillance. The level of surveillance can be 
increased when the following conditions have been met for a sampling period: 


(1) The preceding surveillance period (last month's inspection) 
has been unsatisfactory (ODR exceeds AQL). 


(2) Normal sample size is being used. 
(3) The TRCO and the SCM agree to increased inspection. 


(4) Use the Sample Size shown in Table III, Appendix E, (or go 
to 100 percent inspection) 


(5) Return to Normal Sample Size if after one month the ODR is 
less than the AQL. 


dad. When the Contractor is responsible for failing to meet the 
limits of satisfactory performance (the AQL), the SCM issues a Contract 
Discrepancy Report (CDR) to the Contractor (discussed in 6.6.2 below). if 
the failure is serious enough, issue the CDR at the time of the 
unsatisfactory performance rather than at the end of the month. 


e. If the Contractor does not achieve satisfactory performance in 
that specific service by the end of the next month, the SCM issues another 
CDR, and the ROIC may call in the Contractor Representative for a personal 
review of the problem. 





f. If a third CDR must be issued, the ROIC should consider issuing 
a cure notice. (However, a cure notice can be issued sooner, if necessary.) 


g. Depending on the Contractor's overall performance, the 
Government may issue a Show Cause letter if the reply to a cure notice is 
unsatisfactory. (EFD 02 approval is required.) 


6.6 TAKING ACTION. The QAE may check the Contractor's performance and 
document any non-compliance, but only the OIC may take formal action against 
the Contractor for unsatisfactory performance. This section lists the 
normal steps to be taken by contract administration when the QAE reports 
these deficiencies. The actions listed are not hard-and-fast rules and 
represent a minimum level of action. More serious action can be taken 
sooner. 


6.6.1 Deductions. NAVFAC policy is that deductions will be made for each 
observed and documented defect. Extrapolated deductions based on random 
Sampling will not be used. 


a. The QAE makes a recommendation on the amount of payment 
deductions to be made based on documented deficiencies multiplied by the 
price shown in the Schedule of Deductions, or in the Items of Bid for 
indefinite quantity work items, and the amount of liquidated damages shown 
in the "Consequences of Contractor's Failure to Perform" clause. 


b. When the Contractor's performance is unsatisfactory, i.e., 
exceeds the AQL as defined in the QA Plan and a formal action is indicated, 
the QAE, the TRCO, the SCM, the ROIC, and the Contract Specialist meet to 
determine what action is appropriate. 


c. If a decision is reached to not take a nonetary deduction, the 
reasons are to be documented. The ROIC must indicate agreement with the 
decision by signing the decision documentation. 


ad. Example Deductions: 





(1) Example 1. The QAE has conducted surveillance of a 
contract requirement using planned sampling supported by validated customer 
complaints. At the end of the month results are: 


© Number of inspections conducted - 100 

Oo Number of defects found through planned inspection - 3 
o Number of customer complaints - 15 

Oo Number of complaints validated - 1l 


(a) The Contractor was not given the opportunity to per- 
form the work due to time constraints. 


(b) Observed Defect Rate (ODR) is 14 (3 + 11). Assuming 
AQL was 10 defects/month, the Contractor performance is POOR. A CDR should 
be issued. 
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(c) The established price for each unit of work is $10 
(taken from the Schedule of Deductions). Recommended deduction is $140 plus 
an additional 10 percent for administrative costs which are allowed under 
the "CONTRACTOR's CONSEQUENCES OF FAILURE TO PERFORM" clause of Section 
00004. Total amount recommended for deduction is $154. 


(2) Example 2. Random sampling has been the method used to 
monitor the Contractor's performance. Results at the end of the month are: 


o Number of inspections conducted - 87 

Oo Number of defects found through random sampling - 6 
Oo Number of customer complaints - 5 

o Number of complaints validated ~- 2 


o Number of defects documented 
by Unscheduled Inspections - 2 


(a) The Contractor was not given the opportunity to 
perform the work due to time constraints. 


(b) The ODR is 6.9 percent (6 divided by 87). NOTE: The 
2 defects identified by customer complaints and the 2 day unscheduled 
inspections are not included in the ODR computation. 


(c) The established price for each work occurrence is $15. 
Recommended deduction is $150 ($15 X 10 defects; all defects are used for 
deduction calculations), plus $15 for administrative costs. (The 
"CONSEQUENCES OF CONTRACTOR'S FAILURE TO PERFORM" Clause, Section 000903 
allows an additional 10 percent for defective work that is either 
re-~performed by the Contractor or left unperformed.) Total amount 
recommended for deduction is $165. 


(3) Example 3. The QAE has used 100 percent inspection to 
evaluate the Contractor's performance. At the end of the month results are: 


Oo Number of inspections conducted - 85 
o Number of defects found - 7 


o Number of defects corrected by 
Contractor - 4 


o Number of defects corrected by Government 
Employees ~- 3 


(a) The nature of required work was such that it could be 
re-performed, and it was necessary that all work be accomplished during the 
month. 


(b) The ODR is 7. Since this work was very important to 


the activity's mission the AQL was set at zero. The Contractor's overall 
performance is POOR. A CDR should be issued. 
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(c) The established price per unit of work is $500. The 
recommended deduction is $2000 based on the following: 


- Work re-performed by the Contractor is subject to a 
10 percent deduction for administrative costs as specified in the 
"CONSEQUENCES OF CONTRACTOR'S FAILURE TO PERFORM" clause, Section 00003. 
(In this case administrative cost is taken to mean the cost of 
re-inspection; therefore, the Government must be able to show that 
re-inspection was performed.) This amounts to $200 ($500 x 4 x 10 percent). 


- Work performed by Government employees is subject 
to 20 percent deduction to reflect the cost of the liquidated damages as 
specified in the "CONSEQUENCES OF CONTRACTOR'S FAILURE TO PERFORM" clause, 
Section 00003. If the work in question was bid lump sum, and the 
Contractor's invoice reflects that work, the cost of that work must be 
deducted. This amounts to $1800: $300 ($500 X 3 x 20 percent) plus $1500 
($500 X 3) for the cost of the work. (NOTE: If the work in question was 
bid on an indefinite quantity work item and the Contractor's invoice did not 
reflect the work in question, only $800 would be deducted for that work 
performed by Government employees.) 


6.0.2 Contract Discrepancy Report (CDR). CDR‘'s are written to identify 
documented cases of poor Contractor performance. The Contractor, upon 


receiving a CDR, must identify, in writing, how future occurrences of the 
problem will be prevented (i.e., the corrective action he intends to take). 
Based on the Contractor's response, the Government may or may not take 
further action. 


a. When the Contractor's overall performance for any given contract 
requirement is classified as "poor", the QAE should recommend to the SCM 
that a CDR (Figure 6-3) be issued. 


b. The QAE is responsible for identifying the problem that caused 
the poor performance. This information is not relayed to the Contractor. 
The OAE will use this information ‘to evaluate the Contractor's response. 

The Contractor should have a Quality Control Program that gives him feedback 
on his performance. 


c. If the Contractor's response is lixely to correct the probien, 
the QAE should recommend to the SCM that further Government action is not 
required with the exception of an increased level of surveillance. If the 
response is not likely to correct the problem, the QAE should identify why 
it is not adequate and recommend further action to be taken by the 
Government. 
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1. CONTRACT NUMBER: 





2. TO: (Contractor & Manager's Name) 3. FROM: (Name of SCM/ROIC) 





4. DISCREPANCY OR PROBLEM: (Describe in detail: 


Include reference to 
SOW Directive: 


Attach continuation sheet if necessary.) 





5. SIGNATURE: (SCM/ROIC) 6. DATE: 





~~ 


eros (Contracting Officer) FROM: (Contractor) 


8. CONTRACTOR RESPONSE: (Contractor's proposed solution to correct future 
occurrences of the problem, use Continuation Sheet if necessary) 





9, SIGNATURE: (Contractor Representative) DATE: 








10. GOVERNMENT EVALUATION: 


(Is the Contractor's response a viable solution 
to the problem?) 





11. GOVERNMENT ACTIONS: (Does the Government accept, propose modification, 
Or reject the Contractor's response?) 





12. CLOSE OUT: 


QAE Signature: Date: 
SCM Signature: Date: 
ROIC Signature: Date: 


FIGURE 6-3 
Contract Discrepancy Report 
(Sample) 





APPENDIX C 


SAMPLE QUALITY ASSURANCE (QA) PLAN 
QA PLAN FOR SCHEDULED INDUSTRIAL SOLID WASTE COLLECTION 


1. CONTRACT REQUIREMENT. Clause 3.3 of Section 00005 specifies the re- 
quirements for scheduled industrial solid waste collection. The contractor 
is required to establish collection schedules. Collection standards are: 


- Scheduled pick-ups are made within four hours of the established 
time. 

- Pick-ups are made outside of normal rush hours (or other site speci- 
fic time). 

- Noise levels not to exceed 90 dBA as measured on a general purpose 
sound level meter, (ANSI standard $1.4-1961). 

- No more then five pieces of debris within 15 yard radius of con- 
tainer after pick-up. 

- Containers re-positioned within five feet of specified location. 


2. PRIMARY METHOD OF SURVEILLANCE. Random sampling supported by customer 
complaints. 


3. ACCEPTABLE QUALITY LEVEL (AQL). 10 percent. 


4. QUANTITY OF WORK. Appendix H to the contract specifies 85 sites for 
scheduled industrial solid waste collection. The contractor supplied’ col- 
lection schedule specifies 50 of these sites for weekly collection and the 
remaining 35 sites on a twice weekly schedule. This gives a total of 480 
work occurrences per month ((50 X 4) + (35 X 4 X 2) = 480). 


5. LEVEL OF SURVEILLANCE. Normal surveillance (Level II) is recommended 
initially. If contractor performance is good (i.e., Observed Defect Rate 
(ODR) is 1/2 the AOL, or less), for 2 months in a row, then reduced surveil- 
lance (Level I) should be considered. 


If the Contractor's performance is questionable for the past month (i.e., 
ODR greater than the AQL but less than or equal to 1.5 times the AQL) and 
rednced surveillance was used, return to normal surveillance. 


If Contractor performance is unsatisfactory in any month (i.e., ODR is 
greater than 1.5 times the AQL), then increased surveillance (Level III) 
should be implemented for the following month. 


6. SAMPLE SIZE. Using the Sample Size Tables, the following sample sizes 
are required for an AQL of 10 percent and a population of 480: 


LEVEL OF SURVEILLANCE NO. PER MONTH NO. PER WEEK*® 
Normal (Level ITI) 109 28 
Reduced (Level I) 34 9 
Increased (Level III) e 178 45 


*Assumes four weeks per month. 





7. SAMPLE SELECTION PROCEDURE. Since random sampling is the method of \ 
evaluation to be used, sample selection will be by a random process. The ey 
selection process is as follows: 


a. Collection sites are numbered from 1 to 120. 


b. On a monthly basis select four sets of random numbers of sizes in- 
dicated above (i.e., 28 for Level II surveillance). Each set of numbers may 
have duplicate numbers selected for those sites on a twice weekly collection 
schedule. Numbers may be duplicated between sets. 


c. Each set of numbers selected will be matched against the numbered 
sites. These sites are the collection sites to be monitored for each re- 
spective week. 


d. Weekly evaluation worksheets are prepared for selected sites. 


8. EVALUATION PROCEDURES. The QAE will visit each selected site on the day 
collection is scheduled. Site visits, in general, should be conducted four 
hours after scheduled pick-up time. The QAE should arrange to be at the 
site at the time of pick-up if there have been problems reported with re- 
spect to debris or noise. 


9. ANALYSIS OF RESULTS. At the end of the month, the QAE is to count the 
number of pick-ups classified as unsatisfactory and compute an overall defect 
rate for the month. The Observed Defect Rate (ODR) is the number of unsatis-~ 
factory pick-ups divided by the sample size. 


(Total # U's) 
eee (Total ¥% Us + S‘s) Be 


Since this OA Plan is based on random sampling, unsatisfactory pick-ups 
detected by customer complaints cannot be used in computing the ODR. Un- 
satisfactory pick-ups detected by customer complaints will be used in deter- 
mining a course of action when the contractor's observed performance is 
questionable (i.e., ODR greater than the AQL but less than 1.5 times the 
AQL) . 
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APPENDIX D 


RANDOM SAMPLING 


1. GENERAL. Random sampling is a surveillance method based on statistical 
theory. The key element of random sampling is that each and every occurrence 
of work has an equal chance of being evaluated. In order to achieve the 
desired end results, Knowledge of the Contractor's overall performance based 
on evaluation of only part of the work, surveillance by random sampling must 
be applied properly. Key elements of random sampling are: 


a. Sample size is specified for a given population to achieve a pre- 
determined level of statistical accuracy. 


b. The sample of work occurrences selected for evaluation must be se~ 
lected by a random process in which each occurrence has equal chance for 
selection. 


c. Once an evaluation schedule has established, it must be followed 
through the surveillance period (i.e., monthly schedules). 


d. Surveillance data gathered by other methods (i.e., customer com- 
plaints and unscheduled inspections) can not be combined with data gathered 
by random sampling. 


e. Assessment of ‘the Contractor's overall performance, projected from 
the observed condition of the sample, will always have the potential to be 
in error. Statements as to overall performance should be stated as "The 
Contractor's overall defect rate is in excess of X %". (Where "X” equals 
the observed defect rate minus one half the AQL ~ ex. ODR (9.2%) ~ 1/2 AOL 
(10%) = 4.2%.) 


2. MECHANICS CF RANDOM SAMPLING. Random sampling is a structured approach 
based on statistics to contract surveillance. As such, there is a set pro- 
cedure in its application. The mechanics of applying random sampling are as 
follows. 


3. POPULATION. The total number of work occurrences for a given function 
that are to be performed during the surveillance Deriod must be Known vi 
accurately estimated. The Inventory of Service Worksheet is used to deter- 
mine population size. 


a. When work is scheduled, population size is easy to determine. 
Example: Activity X has 80 dumpsters, 70 are emptied weekly and 10 
are emptied twice a week. The population, total number of work occurrences 
per month, is 360 (70x4) + (10x4x2). 


b. When work is unscheduled, population size must be estimated. 


Example: The number of service requests for the past six months 
were: 
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Jan 321 


Feb 301 
Mar 295 
Apr 337 
May 340 
Jun 320 
Total 1914 


The average number of service calls per month has been 319 (1914/6). This 
would be the expected population for service calls for next month unless 
there is some known reason to expect a change. 


4. SAMPLE SIZE. Sample size requirements are based on AQL, population size, 
and level of surveillance. Sample size tables are used to determine sample 
size. Tables for normal surveillance, reduced surveillance, and increased 
surveillance are attached. 


a. Select the table with the desired level of surveillance (Tables I, 
Thor ILL). 


b. Select the column with the required AQL (.05, .10, .15, .20, or .25). 


c. Select the row that is closest to the population size, preferably 
the next largest entry. 


d. The number indicated by the row and column selection is the sample 
size required for surveillance in one surveillance period. 


(1) Daily surveillance requirements will be determined by dividing 
the required period's sample size by the number of days that surveillance is 
to be conducted. 


(2) Weekly surveillance requirements will be determined by dividing 
the required sample size by the number of weeks in a period. 


(3) When computing weekly or daily sample sizes, always round up to 
the next whole number (e.g. 45 monthly samples required and 20 work days per 
month results in a daily sample size of 3 - (45/20) = 2.25 and rounded up 
results to 3). 


5. Sample Selection. The final thing to be decided in sampling is how the 
Sample will be drawn. The objective in the method is to insure that the 
sample is random (that is, that all services have an equal chance of being 
selected). To achieve random selection, use a random numbers table as ex- 
plained in the following examples. (A random numbers table, Table IV, 1s 
attached). Most items will fall into one of these examples. 


a. Use Of The Random Numbers Table. The random numbers in Table IV are 
arranged in groups of two. 


(1) To use the table, begin by picking at a random a group of jaun- 


bers on any page of the table. This is usually done by closing the eyes and 
pointing with a pencil or finger to some initial group. 


aoe. 





(2) To identify additional random numbers, follow a pattern. Go 
along a given line to its end and then along the next line to its end and so 
on through the table until enough numbers have been selected or until the 
table ends. 


(3) If the table ends and there are still more numbers to select, 
go back to the beginning of the table and continue using the same pattern. 
Use various patterns alternately, for example, use lines for one sample, use 
columns for the next sample, and use a diagonal pattern for the third sample. 


b. How To Use the Random Numbers Table To Identify a Random Sample of 
Consecutively Numbered Work orders. Suppose one has to identify a random 


sample of 97 work orders for evaluation. (Sample size is based on a popula- 
tion of 319 using normal surveillance.) This can be done at the beginning 
of the month (before the work orders are written) or at the end of the month. 


(1) If there are, or might be, 319 consecutively numbered work 
orders to select from, then one begins by listing the lowest work order nunm- 
ber (known or projected). This could be #001, or possibly 443, or any other 
sequentially assigned number. List the highest work order number (known or 
projected); in this case, it could be #319 or 762. For this example, use 
work orders numbered #443 to 762. 


(a) Select 97 three digit numbers from Table IV using a con- 
sistent pattern. 


(b) If random numbers selected are not between 443 and 762, 
discard the number outside the designated range and select a new number. 


(2) For example, using the initial entry on Table IV we would select 
number 441. This number is too low. The next number, going down the column, 
is 343 again too low. The third number selected is 749. This number falls 
in the range of work order number (443-762) subject to inspection. So work 
order number 749 is selected to be inspected. The next work order number 
selected is 523. 


(3) This process would be continued until three work orders are 
selected. 


c. How To Use The Random Numbers Table To Identify A Random Sample From 
A Grouv of Items. If a number of items need to be sampled that are not con- 


secutively numbered, the simplest solution is to list the identifiers, for 
all the items in a column, on a piece of lined paper. 


(1) Next, number the lines consecutively, beginning with the number 
one. Now use the random number table to draw the sample from the line num- 
bers. A selected line number leads to the identifier located on that line, 
and that identifier tells which item to sample. For example, if one chooses 
to sample a set of work orders with attached sales slips, one is not going 
to have to have a set of consecutively numbered work orders because not every 
work order has a sales slip attached. 
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(2) List the work orders with sales slips in a column, number each 
line in the column, and randomly select enough line numbers to make up the 
sample. 

d. How To Use The Random Numbers Table To Identify a Random Sample of 
DayS. Suppose one wants to identify four days in the month on which to sam- 
ple something. The days of the month can be numbered 01 to 31 (or less, as 
appropriate). 


(1) It is best to use a starting point different from the one used 
in the previous example. For the purpose of this example, it is being used 
again. 


(2) One can move down the column from number to number until the 
first number between 01 and 31 is spotted. In this case, it is 22. Thus 
the 22nd day of the month is selected for sampling. 


(3) Continuing in tnis fasnion, one discovers that 11 is the next 
number selected. This number is disregarded. Proceed in this manner until 
the four days for sampling have been identified. In our example, the 4 days 
selected would be 22, 11, 10 and 24. 

(4) If it is not desirable to sample on weekends, discard those 
days selected that happen to fall on a weekend and continue that selection 
until the proper number of days has been selected. 


e. How To Use the Random Numbers Table To Identify a Random Sample of 
Times of Day. If one wants to select random times of day to sample a ser- 


vice such as taxi or bus service, use the 24 hour clock. 


(1) I£ there are any constraints during each 24-hour period, take 
them into consideration. For example, suppose that base bus service opera- 
tes between 0700 and 2300. Convert these times to minutes (e.g. 0700 = 0, 
0410 = 130, 1215 = 315, etc.) Again, using Table IV and selecting three 
digit numbers and proceeding across the line from the initial number, one 
comes to 441, or 1421 hrs, as the first random time. 


(2) The next random number is 343, or 1243 hrs. The number is good 
and so one schedules an observation for 1243 hrs. 


(3) Proceed in this manner until the desired number of sample times 
have been identified. 


£. How To Insure Variety in the Use of the Random Numbers Table. The 
use of variety in the random number table ensures that detectable patterns 


do not occur. 


(1) Success in uSing the tables requires consistency but also vari- 
ety. The above information should ensure that the tables are properly used 
and that the sample is randomly drawn. 


g. Other Random Numbers Generating Methods. The use of a hand held 


calculator with a random number generating capability is an alternative to 
the use of random numbers table. Using this type of calculator the QAE would 
enter the minimum value and maximum value and numbers generated would always 
be within the desired range. 
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TABLE D=-1 
SAMPLE SIZE REQUIREMENTS FOR REDUCED SURVEILLANCE 
(SURVEILLANCE LEVEL [) 


POPULATION AQL 

SIZE .05 .10 .15 .20 
50 31 a 16 ll 
75 34 25 17 13 
100 44 27 18 13 
125 49 28 19 14 
150 52 30 20 14 
175 55 30 20 14 
200 57 31 20 14 
225 59 32 20 14 
250 60 32 21 14 
275 61 32 21 15 
300 63 33 met 15 
325 64 33 21 15 
350 64 33 21 15 
375 65 33 zi; 15 
400 66 34 21 15 
425 67 34 21 15 
450 67 34 21 15 
475 68 34 21 15 
500 68 34 21 15 
550 69 34 22 15 
600 70 34 22 15 
650 70 35 22 15 
700 71 35 22 15 
750 7 35 22 15 
800 72 35 22 15 
850 72 35 22 15 
900 73 35 22 15 
950 73 35 22 15 
1000 73 35 22 15 
1100 74 35 22 15 
1200 74 35 22 15 
1300 74 36 22 15 
1400 75 36 22 15 
1500 75 36 22 15 
1600 75 36 22 15 
1700 75 36 22 15 
1800 76 36 22 15 
1900 76 36 22 15 
2000 76 36 22 15 
2500 76 36 22 15 
3000 a2 36 22 15 
3500 D7 36 22 15 
4000 a 36 22 15 
4500 77 36 22 15 
5000 78 36 22 15 
6000 78 36 22 15 
7000 78 36 22 15 
8000 78 36 22 15 
9000 78 36 22 15 
10000 78 36 22 15 
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TABLE D-2 
SAMPLE SIZE REQUIREMENTS FOR NORMAL SURVEILLANCE 
(SURVEILLANCE LEVEL ITI) 


POPULATION AQL 

SIZE 05 .10 215 . 20 
50 43 37 32 27 
75 60 49 40 32 
100 76 58 46 37 
125 89 66 51 40 
150 101 72 54 42 
175 aa 78 57 43 
200 121 82 60 45 
225 129 86 62 46 
250 137 89 63 47 
275 144 92 65 48 
300 151 95 66 48 
325 157 $7 67 49 
350 162 99 68 49 
, 375 167 101 69 50 
400 ie 103 70 50 
425 176 108 71 51 
450 181 106 71 51 
475 184 107 72 51 
500 188 109 72 52 
550 195 111 73 52 
600 201 113 74 52 
650 206 114 75 53 
700 211 116 75 53 
750 215 117 76 53 
800 219 118 76 54 
850 222 119 a0 54 
900 226 120 77 54 
950 229 121 78 54 
1000 231 ie 78 54 
1100 236 123 78 55 
1200 241 124 79 55 
1300 244 125 79 55 
1400 248 126 80 55 
1500 251 127 80 55 
1600 253 128 80 55 
1700 256 128 80 55 
1800 258 129 81 56 
1900 260 129 81 56 
2000 262 130 81 56 
2500 269 131 82 56 
3000 274 132 82 56 
3500 277 133 82 56 
4000 280 134 83 57 
4500 282 134 83 57 
5000 284 135 83 57 
6000 287 135 83 57 
7000 289 136 83 57 
8000 290 136 84 57 
9000 291 136 84 57 
10000 292 137 84 57 
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TABLE D-3 
SAMPLE SIZE REQUIREMENTS FOR INCREASED SURVEILLANCE 
(SURVEILLANCE LEVEL IIT) 


POPULATION AQL 

SIZE 05 eu PS) 20 
50 47 43 39 35 
75 67 59 52 46 
100 86 74 63 54 
125 104 87 72 60 
150 21 98 30 65 
175 136 108 86 69 
200 Lo 116 92 73 
225 164 124 97 76 
250 177 132 101 79 
275 189 138 105 81 
300 201 144 108 83 
325 2il 150 lll 85 
350 222 155 114 86 
375 231 159 116 88 
400 241 164 119 89 
425 249 168 121 90 
450 258 172 123 91 
475 266 175 124 92 
500 273 178 126 93 
550 288 184 129 95 
600 301 189 raz 96 
650 313 194 134 97 
700 324 198 136 98 
750 334 202 138 99 
800 344 206 139 100 
850 352 209 141 101 
900 361 212 142 101 
950 368 214 143 102 
1000 376 ZL 144 103 
1100 389 221 146 103 
1200 401 225 148 104 
1300 411 228 149 105 
1400 421 231 150 106 
1500 429 234 Se 106 
1680 437 236 1S2 107 
1700 444 238 153 107 
1800 451 240 154 107 
1900 457 241 155 108 
2000 462 243 155 108 
2500 485 249 158 109 
3000 501 253 159 110 
3500 5L3 256 161 lll 
4000 523 259 161 lil 
4500 530 260 162 lll 
5000 537 262 163 112 
6000 546 264 164 ne 
7000 554 266 164 L12 
8000 559 267 165 113 
9000 563 268 165 113 
10000 567 269 165 113 
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31 
39 
45 
50 
53 
56 
58 
60 
62 
63 
64 
65 
66 
67 
68 
69 
69 
70 
70 
71 
72 
73 
73 
74 
74 
12 
75 
75 
76 
76 
aT 
77 
77 
Vd 
78 
78 
78 
78 
78 
79 
80 
80 


80 
80 
81 
81 
81 
81 
81 





44 
34 
74 
22 
40 
D2 
37 
Hel! 
10 
67 
98 
24 
Oe) 
74 
Si 
42 
06 
92 
91 
68 
59 
39 
67 
61 
78 
a3 
37 
62 
29 


04 
95 
86 
59 
05 
TW 
80 
13 
67 
72 
28 
69 
at 
89 
58 
50 
49 
aE 
34 
42 
84 
Zo 
04 


15 
80 
80 
61 
96 
76 
20 
55 
79 
28 
75 
Z3 
63 
19 
35 
66 
63 
96 


37 
13 
sz 
31 
37 
91 
71 
48 
11 
97 
85 
10 
43 
so 
60 
35 
17 
32 
78 
65 
a3 
85 
40 
03 
55 
82 
73 
72 
01 
46 
64 
72 
94 
51 


TABLE D-4 


SHORT TABLE OF RANDOM NUMBERS 


63 
24 
65 
86 
03 
56 
a3 
48 
34 
47 
09 
80 
38 
54 
81 
da 
LS 
89 
52 
47 
13 
33 
12 
56 
00 
09 
63 
Mey 
03 
89 
15 
97 
35 
41 
54 
66 
98 
13 
45 
26 
94 
44 
1 
22 
14 
SL 
10 
65 
84 
22 
44 
80 
61 


55 
33 
07 
38 
47 
15 
17 
B84 
54 
89 
18 
64 
27 
18 
16 
91 
03 
B0 
13 
53 
80 
46 
60 
90 
63 
78 
89 
23 
13 
07 
12 
18 
48 
31 
03 

27 
00 
39 
36 
04 
ll 
94 
25 
45 
02 
07 
20 
02 
74 
75 
25 
55 
34 
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87 
81 
71 
33 
24 
47 
82 
74 
71 
ll 
54 
34 
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28 
61 
03 
V2 
87 
04 
90 
42 
58 
13 
12 
1S 
08 
92 
29 
20 
17 
98 
85 
68 
00 
98 
85 
40 
66 
77 
62 
58 
60 
74 
12 
01 
81 
65 
85 
Ze 
81 
78 
31 
63 


a, 
67 
30 
71 
60 
75 
24 
36 
33 
65 
34 
20 
78 
a2 
97 
63 
38 
14 
82 
05 
20 
aS 
57 
39 
63 
03 
59 
Zo 
86 
30 
84 
7 
18 
69 
96 
rr 
92 
82 
96 
81 
78 
82 
7, 
49 
05 
26 
28 
69 
02 
74 
22 
63 
87 


33 
28 
01 
13 
09 
78 
cS 
oe 
89 
65 
68 
23 
88 
33 
25 
48 
O01 
25 
23 
90 
63 
34 
63 
03 
74 
07 
14 
13 
22 
Zo 
60 
13 
36 
63 
76 
oS 
oF 
Ol 
46 
15 
45 
94 
78 
82 
77 
25 
44 
56 
57 
91 
07 
27 
44 


29 
ll 
84 
oe 
21 
73 
90 
67 
74 
20 
02 
46 
84 
69 
14 
46 
58 
49 
00 
34 
03 
03 
78 
56 
58 
71 
a2 
85 
45 
89 
93 
08 
57 
77 
a 
56 
51 
28 
Zu 
35 
36 
93 
34 
71 
94 
45 
63 
88 
77 
48 
62 
91 
13 


45 
34 
47 
72 
21 
78 
80 
iS 
68 
42 
54 
L5 
42 
98 
78 
44 
25 
25 
26 
87 
24 
59 
Li 
49 
69 
‘io 
19 
33 
So 
39 
68 
00 
09 
01 
77 
= 
52 
67 
14 
79 
34 
98 
54 
57 
65 
49 
95 


47 
46 
17 
70 
50 


00 
79 
45 
08 
18 
19 
87 
20 
48 
23 
87 
36 
32 
96 
21 
01 
37 
94 
36 
48 
64 
28 
80 
32 
03 
32 
17 
35 
98 
80 
16 
50 
62 
89 
34 
36 
83 
aL 
Bo 
26 
45 
O01 
45 
33 
57 
61 
86 
23 
33 
18 
2 
74 
56 


31 
26 
89 


00 
87 
32 
15 
23 
96 
23 
10 
00 
74 
Ze 
33 
66 
62 
47 
25 
12 
a7 
10 
a2 
51 
25 
22 
07 
64 
98 
87 
ad 
40 
94 
80 
48 
14 
75 
56 
oo 
a1 
48 
04 
28 
70 
22 
5 
64 
i 
34 
34 
13 
48 





APPENDIX E 


EXTRAFOLATED DEDUCTIONS, BASED ON SAMPLING TECHNICUES 


a Mae ge Tat ee a ae ae ee ee ee ee 
= | 
| Givens | 
| Memes ceors LOe Size :; 450 units; sample ;: 50 units 
Ten defectives were found in the sample. | 
| Maximuad contract payment pér month: $10,000 
| Fayment percentage for the service (from PRS): <x S&S 
| Maxintur payment for acceptable service: $500 | 
| Umgeceesace ($500 divided Ey 450 twnits) ft, 1 | 
10 defectives exceed a reject number of 8: 
| Cefective percentage in sampiz: 20% | 
| Fercéentage cf sample Ecund accentable: 80% | 
| Credit for corrécted samples: neue ! 

Acceptakle percentages 82.2h 
| Payment for accéeprabie service: $411 
Ee eee 


Fagure £.1 Air Ferce Deductions Under Random Sampling. 


This figure shows how deductions would normally be made 
C is 


Weemerae CGivtlactor exceeds the AQL for any month's pericr- 
Mance, This example illustrates how credits would be appiisd 
wnen work is allowed +o be reperrormed. Net NOue TcDePlor= 
mance, zhe above final payment wouid inscead be 34090. 
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Figure E.2 





AQL : 5%; lot size : 50 units; defectives 
Maximum con*ract payment per month: 


Maximum payment cercentage for 
Specific ssrvice; 


Maximum payment for acceptabl 
service: 


Unit price ($400 divided ty 50) 
5 defectives exceed a reject number of 3: 
Fercentage cf ct found acceptable: 


Fayment for accertable performancs:: 


PEEP TEP Ge aa a ES a: 





This figure illustrates how deducticns 


7S ee TS eee ee ee me 


O 
© 
L 3 | 


Alr Force Deductions Baséd on Planned Sampling. 


are made when 


rlanned sampling procedures are utilized. Deducticrs would 


hot 


be taken if the defectives cbserved were 


less than the 


Geject number of three. When the AQL for a month's voericr- 


mance is 


exceeded, the centractor should be notified by 


means of a discrepancy report. If successive discrepancy 


Eeports must be prepared, a Show Cause or Cure 


required. 
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SS ee aan ee a 
1 | 
{ Rit wt ems are sutject to 100% inspection. | 
| Cosz of required services: $5000 | 
| Total number of work units(loz size): 100 | 
| Unit price ($5000 divided by 100 units): $59 | 
Sampie size: 100 | 
| Cbhserved nonconforming items: 5 | 
| laeetS cesomked at government option: 3 
| Weer eS SCE Creditabis Coz paynenc: 3 | 
| Tetal units creditable for paymnen=: at | 
| Mereeneags cE centract pried due: 972 | 
| Collar value due: $4850 | 
| Liguidated damaces at 10% of | 
memeceneOl@ong Service (210 <6 «% 350) : $3 
Actual amount paid: $4820 | 
| 
a Z ge j 





Fagure £.3 NAVFAC Fayment Under 100 Percent Inspecticn. 


This figure illustrates NAV 
uUme@er 100% jinspecticn. Liquidate 
[om Or “<he value of initial ob 
Sapié. Izems mav ke zesutrn 


payment. 


2n0) 





—— = _ — 
| Fandcm sampliag is being utilized. 

| All aefectives are reworked. | 
| Ccest of reguired services: 55000 | 
| Total work units (lot size): 1000 ! 
i Uget Peace ($5000 divided by 1000 unic<s) » | 
| Sample size; 122 | 
| Observed nonconforming: 14 | 
| ercentage nonccnforming: G4 | 
| BOIL : 10% | 
| Units reworked at government option: 11 
| Total units creditable for paymencz: 1000 
| Bemceacaege Of ccntract price due: 100% | 
| Cellar vaiue due: $5000 | 
| eee iroruine sereice (.46 aXe oO es oD | 
| Actual amount paid: $4994.25 ! 
| ee eal 


Figure E£.4 NAVPAC Random Sampling Deductions Where ACL is 
Not Exceeded. 


Under this scheme, randcm sampling procedures are being 
utilized. As before, liquidated damages are asse 
all nenecnfcrming items that are observed. [fa 
reworked, as in this case, fuli paymeat Less an 


deémages is made. 





— ee ee ee eee ee a ee ee ee ee ee Se a ee ee ee ee ee > ee se ee ee eee eee 


Randcm sampling frocedures ars being utilize2d. 


cece cee SOE ines eens caeipamns NEED wage 


| 

| 

| Geet Gt sequired Servics $5900 

| Meedivna@mber of work units (lot size): 1000 
Unit price ($5000 divided by 1000) £5 

| Sample size: 22 

| Chserved norconfcrminrg units: 11 
| Trreshclid (AQL): 10% 
| Cbserved percentage nonccnfornirg: oe | 
| Units reworked (at Government option): 6 
| Umeusenet creditable fcr cayment: 5 | 
Pocageuhits Creditable for paymenc: 995 
| Fercentage cf centract price due: Jena | 
| Dceilar value due: £4975 
| Liguidated damaces (10% cf nonconforming | 
| seems, rb .10 x 11 x $5) San 
| Actual amount paid: $4969 .5 ! 
(a aes! s<_ ea 





Figure E.5 NAVFAC Deductions when defectives are less than 
AQL and some are reworked. 


This figure shows that random Sampling is the inspec- 


mean and that not all of the units found detective were 
rTewerked. Thus, deductions for liquidated damages and defec- 
RaSeerce-Guse Le 20t 


tive cpsérved sample items are taken. MT: 
Ue li zed Ey the Air Fcrce. 
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: 
| 








et Seay en eee ee aee nS ee ee ee 


Randcm sampling procedures are being utilizec. 


Some defectives are reworked in the allotted ri 


Liguidated GCamaces at 10% of the value 
Of ncrcontforming services based on 
eeecepOra-ea fercentages (2.14 x 100@P x 5 x .1):370 


Rozval amuourt paid: $4320 


| | 
| 7 
| | 
| } me. | 
Cest of required services: SoUoC. si! 

| fecalenunber of work™units (lot size): 1000 ! 
i Unit price ($5000 divided by 1000 units): $5 | 
| Sample size: 122 ! 
| Cbhservid defectives nonconforming: 17 | 
| Fercentage nonccnforming: 14% | 
| Waceshcid (AQL): 10% 
| Extrapclaticn of defective percentage | 
| Pom@meatn ctOtal acceptabl? units: 860 | 
| Units reworked (at Government opticn): 14 
| Total units creditable for payment: 874 | 
Peeec-ncags Chk CCntract vrice due: 87.4% | 
! Ccllar value due: $4390 ! 
| | 
| | 
! 

| 


L 


ae eee ee ee OR Ss Se ee eae 


Figure E.6 NAVFAC Ceductions where AQL is exceeded and 
some defectives are reworxed. 


Thess illustraticn shows thac when random samrlin 
reveals chserved defect rates in the sample to ke gréater 
shan the specified AQL, liguidated damages and payment 


deducticns use sample defect percentage. detis ha. fezc 


C 
resubmitted are credited only after the extrafo 
iS 


Hor FH 


Maeecns @le complete. Thi e in conce 


. 


able statistical techniques, 


Saewemgwecn-sactcr attention <9 dete 
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SS ee a ee ee eee A 
pe eae ee ee —_— Ee we 22 eee 


Flanned samcling inspection is in effect. 

Cost of required service: $5000 
Total number of work units (iot size): 10C0 
Unit price ($5000 divided by 1000): so 
Sample size (as desired): 122 
Cbserved nonconfcrming units: NZ 


Sic s breworked within allo=ted tine: 


noida Oe iin ol 


. 


Total units creditable for paynent: 9¢4 

Bopecmhtede cL CCntrac= price dues 99.4% 
(Ceduct only fer observed derzects) 

Collar valué due; $4970 

Liquidated damaces at 10% of value of ; 
nenecntorming services: $6 

Actual amount paid: $4964 

_ i ee ee ee ee ee eee 


Figure E.7 oe Teductions under planned sampling proce- 
ures. 


The figure shows the deduction procedures for planned 


sampling. 





AP PENDIX 

INSPECTICN RESOURCE ESTISATION 
LANTDIV 

The three 


Ba=aCurcs ¢€s 


EOLIOW A ag 


workshests are 


<imating werksheets thax 


PE 


WORKSHEETS DEVELOPED EY 


OG 
develcped and 


exampies QAE 


have been 


Seemuldaced by the Atlantic Division, Navel™ Faciligees 
Fogin¢eering Command These worksheets apply for flanned 

andcr and 100 percent inspection me¢eznods and should result 
in fairly accurate estimates of QAE requirements. Aan 
monthly <zotals can be accumulated to obtain annual heur 
tequirements, and thus provide an estimate of perscnnel 
céiling coints that should either be retained or requested. 
This mrethcdclogy weuld provide future benefits in that acen- 
cies might ke akle te accumulate actual hours expended for 


Piepeccicn during 
ae 


wCrk requirements that are 


and thus res 


i¢eems Listed in 
SeGersececnd =C work items that 


requirements Summaries (PRS ‘'s). 


ole 


the performances 


in a manhour eStinax 


are lis 


Demi odors Car cont tacts , 


(ag tacieOae LOG Vare Cus 


performed under the contract 


the contract requirements columns should 


téd on the performance 





LANTNAVFACENGCOMINST 11014.4D 


QAE WORKLOAD CALCULATIONS PLANNED SAMPLING 


INSTRUCTIONS: Fill in the contract requirement, quantity of work and level of 


survelllance from Performance Requirement Summary Table. 
Calculate the number of inspections. 


Using the estimated hours 


per inspection, calculate the number of hours of inspection 


required per month. 
levels. 


Contract Requirement 


Quantity | Level of Number of | Hrs. per 
of Work Surveillance Inspecs. Inspec. 


Zo 


Refer to MO-327 for suggested surveillance 


Monthly 
Hrs. of 
Inspecs. 





a 


Pa LANTNAVFACENGCOMINST 11014.4D 


QAE WORKLOAD CALCULATIONS 100% INSPECTION 


INSTRUCTIONS: Fill in the contract requirement and quantity of work from 


Performance Requirement Summary Table. The 
surveillance is 100%. The quantity of work 
inspections. Using the estimated hours per 
calculate the number of hours of inspection 


level of 

is the nusber of 
inspection, 
required per month. 


Quantity | Level of Number of | Hrs. rer 
Contract Requirement of Work Surveillance Insoecs. Inscec. 


100% 


intemal 
[| 
Daa o te ee) 
one a. 
ae 
eS eee 
i glk Sg ea = 
a 
eo 
pt toe ff 
os ee 


100% 


~ 100% 


a ee 

| oor | 
el I 
en eee ee 
ee! 
Li tite ee = 
Be oO ee 


2A ; 


Monthly 
Hrs. of 
Inspecs.e 





a“, 
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LANTNAVFACENGCOMINST 11014.4D 


- QAE WORKLOAD CALCULATIONS RANDOM SAMPLING 


INSTRUCTIONS: Fill in the contract requirement, quantity of work and the 
number of inspections to be performed from Performance 
Requirement Summary Table. Indicate surveillance level. 
the estimated hours per inspection, calculate the number of 


hours of inspection required per month. 


Using 


Refer to MO-327 for 


recommended surveillance levels and associated sample size 
requirements. 


Contract Requirement 





Quantity {Level of Number of | Hrs. per 
Surveillance Inspecs. Inspec. 
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VARIOUS CA CONTRACT ALCMINISTRATION ORGANIZATIONS STUDIED EY 
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EWC = Public Works Officer 

AEFWO = Assistant Public Works Officer 
Poe@meGnnidce = Facilities Sstvice Contract: Olde ce 
MCL = Maintenance Control Divisicn 
e@lume—-so=-Vice Conttec=. Manage 

FE'S = Blanners and Estimators 

CAE's = Quality Assurance Evaluators 
iieiomomoan: 2at.0n-10Cates es. tse ort ce as 4 
Cempistely centralized activity divisicn. 


em a =, ee oD i ee ee a ce Oe ee cee Se ee nee ce ee mm Me Ee 


ek 


| 
| 
| 
| 
l 








Pigure G.1 Centralized Organization With SCM as Manager. 
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Centralized Organization Located in MCD. 
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| Peeneacal tc Figure 1; additionally: | 

HOUSG = Housing Divisicn | 

{ Seite = SeCCULIty Division 

| iano Fe anevoOrtat2on Divesion 

SCM's supervise a limited number of (er fc) 

| CanSS OCAE*S primarily work for Eunctional 

| Managers. SCM functions more as an advisor. | 
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Figure G.3 Decentralized System With SCM in MCD. 
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Same as in Figures 1 and 3 


Peto vOLGaniZaca Chal fO=mat, mene SCMiss a 
Givesmon head Of the aCt2vicy. Direc. Centrol 
cf QAE'S is exercised by functional managers. 
[mye Seteacts prime tily in an adviscey Gapacity. 
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Figure G.4 Decentralized System: FSC Office is a PWD Division. 





AFIC SUPFLEMENTARY POLICY FOR QAE BRANCH ORGANIZATION 
himcns Appends x, APLG Supplement No. 11s peovided. It 


illustrates amplifying Air Force Logistics Command Guidance 
which pertains to AFR 70-9. 
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DEPARTMENT OF TIE AIR FORCE 
Headquarters Air Force Logistics Command 
Wright-Patterson Air Force Base OH 45433 


AFLC SUPPLEMENT 1 
AFR 70-9 
26 February 1982 


Contracting and Acquisition 


BASE LEVEL CONTRACT ADMINISTRATION 


AFR 70-9. 25 September 1979. is supplemented as follows: 


1c. Where separate contract administration offices exist. the 
term *BCO” in the basic regulation is interpreted as “ACO,” 
except in paragraphs Ie and &d. 


. 6a AFLC/PMM is OPR for the QAE program within 
AFLC. 


6b. All QAEs will attend a phase I QAE training program. 
managed by the local QAE program coordinator. 30 days 
before the date of initial contract performance. If the train- 
mg can't be completed 30 davs before contract start, the 
contract file will contain the documented reason why the 
training couldn't be done within the specified time. Senda 
copy of the documentation to the functional area chief (FAC). 
The training program will be divided into two phases: The 
administrative contracting officer (ACO) is responsible for 
phase I training. which is basically an orientation. It covers 
provisions of individual contracts over which the QAE will 
have surveillance responsibility. Phase II training will be 
conducted using inlormation and materials supplied by HQ 
USAF/RDCL. and will be completed at the first quarterly 
training session ofter appointment. This training is applicable 
to all contracts. The QAE program coordinator will give the 
Cividian Personnel Office a list of names of all the people who 
have sucessfull compieted the Air Force-prepared training 
course. The Civilian Personnel Training Office will: 


(1) Authenticate completion of the training on DD 
Form 1556. Request. Authorization, Agreement, Certification 
of Training and Reimbursement. 


(2) Place a copy of DD Form 1556 in the individual's 
official personnel folder. 


(3) Give a copy to the individual, the QAE program 
coordinator, and the cognizant FAC. 


(4) Keep records on file for persons completing this 
training. 


Supersedes AFR 70-9/AFLC Sup 1, 5 May 1978. 
No of printed pages: 3 

OPR: PMM (W. Ely) 

Approved hy: Col Paul Baldasan 
Writer-Editor: C. Rainey 


r 


7b. Uponreceiptof Letters of Nominationfromthe FAC, the 
commander will appoint QAEs and their alternates in writing. 
A sufficient numher of QAEs will be appointed to make sure 
adequate surveillance is provided at all times. 


Sd. Give copies of the letter of appointment to the 
contracting officer. along with the purchase request for 
reculring contract requirements. If a cost study is being 
conducted, furnish the letter of appointment as soon as the 
QAE is selected. The FAC will senda copy of the letter to the 
QAE program coordinator and the servicing Civilian 
Personnel Office. The Civilian Personnel Office will make sure 
AF Form 1378. Civilian Personnel Position Description, 
reflects duties of employees named as the QAE and alternate 


QAE. 


8j. Develop surveillance plans (SP) for those contracts that 
don't have standard Performance Work Statements (PWS) to 
ensure proper contract surveillance. The contracting officer, 
FAC. and QAE should jointly prepare SPs and revisions to 
them and should tailor them to the individual contract. The 
frequency of QAE surveillance and the typeof inspection (for 
example. random sampling, 100 percent inspection, 
surveillance check-list) for each surveillance area should be 
specified. Do this betore the contract startdate. If experience 
with the contractor shows that extensive surveillance isn’t 
required, the SP may be revised to reduce the level of 
frequency of surveillance. Any revisions to SP require the 
concurrence of the contracting officer. Where SPs take the 
form of checklists, attach them to the AF Form 372, Contract 
Monitonng and Surveillamce Report. and submit to the 
contracting officer according to paragraph 10¢ 


9d. The ACO with the help of the QAE program coordinator, 
will: 

(1) Give guidance to the FAC and QAE as appropriate 
regarding contract interpretation, resolution of problems, 
requirements of this regulation, and other related contracting 
matters. 


Distribution: F:X (HQ USAF/LGP: HQ AFISC/DAP: AUL/SE...........1 ea) 





(2) Indoctrinate the QAE and alternate QAE as to the 
extent of authority. responsibility, and limitation. as outlined 
in the contract and this regulaton. Do this pnor to 
contractor performance. 


9e. Provide copies of contracts and contract modifications to 
the cognizant QAE, as well as the FAC, prior to service start 
dates. , 


9g. Ifthe AFO and FAC agree thata quarterly meeting isn't 
required. document the contract file. When quarterly 
meetines are held. personnel attending will include as a 
mintmum the contracting officer, QAE program coordinator, 


the FAC, and the QAE. 


9h. Furnish the QAE with a information copy of 
correspondence pertaining to contract requirements or 
discrepancies sent to or received from the contractor. 


9k(Added). The local QAE programcoordinator is the focal 
point for the QAE program. The QAE program coordinator 
wil keep a record of all contracts requimng QAEs, and the 
names and training records of QAEs appointed. The QAE 
program coordinator may be required penodically to provide 
mformation on the QAE program to HQ AFLC/PMM. Any 
reporting requirements developed in support of the QAE 
program will comply with AFR 178-7, as supplemented. The 
QAE program coordinator will work closely with cognizant 
contracting officers in making sure QAEs understand the 
responsibilities and the surveillance of specific contracts. The 
QAE program coordinator will make sure records are 
maintained of the conferences held accordingto paragraph Yg. 


10b. The Chief QAE is responsible for certification of 
contractor invoices that services were satisfactory. 


10c. The QAE will maintain appropriate forms (AF Form 
802, Contract Discrepancy Report: AF Form 799. Surveillance 
Activity Checklist) for each contract. The use of AFLC Form 
134, TRCO Daily Log, is authorized only until the present 
supply of the form is exhausted. or until new contracts go into 
effect AFLC Form 134 is not authorized for contracts 
awarded after date of this publication. Type entries or record 
them legibly in ink: 


(1) Enter and sign on the forms. as required hy the 
contracting officer for services performed, a notation of the 
contractor's performance including any deficiencies. For 
services performed on other than a daily basis. enter and sign 
the notation when the contractor actually performs the 


services. Annotate this only when the QAE pertorms an 
Mspection. 
(2) Record any performance deficiencies noted in 


precise. descriptive language. Notify the conuractor’s project 
manager (or authorized representative) and request 
acknowledgement by concurrence or nonconcurrence in the 
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Remarks section of the forms that thedeficiency does exist. If 
the contractor concurs. a statement is required outlining 
actions planned or taken to correct the deficiency and prevent 
its recurience. In cases of contractor nonconcurrence. the 
QAE should immediately contact thecontracting officer. Ifthe 
contractor's project manager (or authorized representative) 
isn't available to discuss the discrepancies. notify the 
contracting officer who. in turn. notifies the contractor (hy 
letter if trme permits) of the deficiencies. When prompt 
corrective action isn't taken or when a deficiency becomes 
more serious, notify the contracting officer and the FAC. 

Document the notification by annotating the AF Form &02. 


(3) Hand-carry, when feasible, the QAE reports for the 
previous month (assembled and sequentially numbered) to the 
contracting officer by the 5th workday of each month. or more 
often if deemed necessary by the contracting officer. The 
contracting otficer will review and initial the QAE reports and 
will then place the reports in the official contract file unless 
the contracting officer determines that the QAE should keep 
the reports. In that case. the reports are furnished to the 
contracting officer for inclusion in the contract file upon 
contract completion. 


10e(Added). 
not: 


Limitations of Authority. QAE personnel! will 


(1) Clarify. make. or infer legal interpretations on the 
scope or intent of the contract. 


(2) Approve contractor procedures unless specifically 
provided by the terms and conditions of the contract. When 
contractually authorized. approval authonty remains subject 
to any limitations the contracting officer may impose. 


(3) Authorize expenditure of funds, except under the 
specific terms of the contract. 


(4) Levy or impose upon contractors any task or permit 
any substitution not specifically provided for in the contract. 


(5) Enter into contractual 
contract modifications. 


agreements including 


(6) Give direction to the contractor or to employees of 
the contractor. 


(7) Offer advice to the contractor which may adversely 
affect contract performance, compromise the rights of the 
Government. provide the basis of a claim for constructive 
change. or impact any pending or future contracting officer 
determinations as to fault or negligence. 


11. Waivers to the requirements ofthis paragraph will 
be in writing, signed by the FAC. Send copies to the QAE 
program coordinator and the contracting officer. 
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12. Send all requests for deviations to HQ AFLC/PMM. 


OFFICLAL 


WILLIAM R. CARROLL. Colonel, USAF 
Director of Administration 


NO 
OD 


JAMES P. MULLINS, General. USAF 
Commander 





ABBREVIATIONS USED IN THE STUDY TEXT 


Abbtreviaticn 


N=76 
AFR 
AFSCAG 
AFLC 
ACL 
ASBCA 
BCE 

BOS 

CA 

CNM 

CNO 

DOD 

BED 

ry 

GAO 

Gis 
LANTLIV 
MCD 

MrO 

er oi 10 5D 
NAVEAC 
NAYSUE 
NCRTBCIV 
NRCC 
Nec 
Oiee © 
Cur 


Description 


OME Circular A-76 

Aix Force Regulation 

Ame BOErGEEService Contract Advisory Grcup 
Air Force Logistics Command 
Acceptable quality level 

Armed Services Board of Contract Appéais 
Bureau of the Budget 

Base Operating Support 

Cecmmercial Activities 

Chief of Navy Material 

Chief of Navy Operations 

Department of Defenses 

Engineering Field Division 

Fiscal Year 

General Acccunting Office 

General Schedui2 Series 

Reven tac) DivVisaon, NAVEAG 
Maintenance Ccntrol Division 
NeSeeree 2CUens Ol Gana lan2oOn 

Maditary Standard 105D 

Naval Facilities Engine¢ring Command 
Naval Supply Systems Command 
NO@rTEhern “Division, WAVEAC 

Navy Regional Contracting Center 
Naval Supoiy Center 

Office of Federal Procurement Policy 


Office of Management and 3Budgst 


Zed 





ONR 
Eis 
FAC 
FWD 
Pus 
QAE 
SOLS 
SCA 
SU ay 


O£fic]= of Naval Research 
Performance Requirements Summary 
Public Works Center 

Putlic Works Deéepartnent 
Performance Work Statement 
Quality Assurance Evaluatcr 
State seca) Oualiay contro. 
Servac=e, COnclace Manager 


Sieter arn pavis7Oone NAVFAc 
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